These are unedited transcripts and may contain errors.

DNS Working Group

Wednesday, 14 May, 2014, at 2†p.m.:

JIM REID: Good afternoon, everybody. I hoped you have enjoyed your lunch break. This is the second session for the DNS Working Group so if you are expecting another discussion about, I think it's address policy you are probably in the wrong room. And I suspect we probably have got a bunch of people downstairs in the overflow room because we are getting fairly full here. There is one or two more seats. Those guys at the back of the door, go downstairs to the overflow room.

Before we get started, just one or two other announcements. If you have got any devices like mobile phones and smart phones and tablets and things, could you please set them to silent mode. We have got two people from the NCC here, one is doing the scribing and another lady is Emile is taking care of the Jabber room for us.

If you are going to come up to the microphones to speak, please remember to state your name and your affiliation because there are people following this in web land and not everybody is going to remember names and faces and voices.

Our first item of business for this afternoon is discussion and report from a little ad hoc group that was looking into the provision of DNS TLD service that the NCC offers and trying to find out some kind of policy framework for that†?? I wouldn't like to call it policy but it's not policy. Peter is going to report to you what has been discussed and what we think might be the way forward.

PETER KOCH: Good afternoon. Thanks, Jim. As Jim said, this is a brief report of the ad hoc group, I will have the names of the people who participated at the end. This is referring back to the discussion that we had in Athens, I guess, and a couple of times before, looking at who is the RIPE NCC actualing DNS secondary service for and Anand mentioned in his report that that is Reverse?DNS and amongst a few other non?noteworthy things, country code top level domains and there has been a discussion that the RIPE NCC may or may not compete with its members or getting us straight from core mission or something like that. So let's have a look at the.

Just a reference to the RIPE NCC activity plan and budget. The URL is up there. I will not read this text, you can do that yourself. That means that, yes, it is in the RIPE NCC's mission to provide this secondary service but in the second part, there is an explicit reference that the policy under which secondary service for cc TLDs is provided is up for review this year and that is why this group came into existence.

It says the RIPE NCC is no longer providing this for well?established cc TLDs, now you can dive into details, when or where is it to decide who is well?established and not so well. One of the basic things to look at is, the basic reason for doing this is, again, not because secondary service is so fancy and colleagues at the RIPE NCC have nothing else to do, but it is to contribute to the stability, security and resilience of this service within the RIPE region reflecting the consumer site and for those TLDs who cannot... afford, we will come to that later, otherwise later. That means developing cc TLDs, developers of cc TLDs, not so much the developing operator and requiring it in the sense they are in the desperate need of having service.

I have two hands to coordinate. So, the group got together and with all that said, the mission, the exerpt from the activity plan and the guiding principle being stability and resilience. This is not going to be a new core service, there is no intent to grow and expand it probably just the opposite, but what was needed, because every now and then with all these new ccTLDs and especially new IDN, once a in a while somebody comes up to the RIPE NCC and asking for service and there is little to decide upon who is eligible and who is not.

So the group agreed to suggest a couple of guiding principles here. First of all, the RIPE NCC should provide this service only for ISO 3166, that is standard two letter country code or what is currently called IDN ccTLDs. So under this clause the RIPE NCC will not engage in secondary service or other DNS service for all this quantity I will I don't know new fancy, Shane knee, TLDs that we are all looking forward to.

Again, stability, resilience and one of the serious reasons behind this is under the ICANN rules, a new operator for a quote commercial TLD or other new gTLDs has to provide a plan for providing the service already upon application, so that is already being dealt with and there should be no extra need for the RIPE NCC to jump in to support the stability.

One thing that has probably not happened in all cases in the past but helps for two reasons, is that there should be a letter exchange and letter is a bit in italics here because it has to be decided afterwards by legal department and others how formal that letter is going to be, with those TLD operators and that should be renewed on a periodic basis. One reason is that the expectations and responsibilities are documented and the service will only be provided on a best effort basis, best effort basis does mean it's a professional service as the RIPE NCC is used to provide, but under specific circumstances they might†?? there might be reasons to give this service less priority than other more core services.

The second reason for having this letter exchange is that this kind of assures that there is a maintenance of the existence of the other party. In some rare cases, operators of TLDs sometimes have vanished or have been hard to reach after a couple of years but with this letter exchange there is an assurance that people remain in existence or at least can be talked to.

In that same sense, the agreement will have a finite lifetime so it's not a subscribe now and receive the service forever. It makes sense probably. And there are three eligibility criteria that the RIPE NCC would apply: The number of registrations should be like reasonable and comply with what was given in the activity plan, so we have refrained from writing explicit numbers here because that is, it's a fishy thing, why 10,000, why not 10,001, it's either the zone is big or small, I can't describe it, but I will recognise it when I see it, kind of thing.

Number of diversity of other name servers, again if there are very diverse and range providing service for that zone the basic reason adding to the stability and resiliency and need wouldn't be given. Same indication from the third criteria here, if somebody is already using commercial services is actually able to pay for it, addressing the concern that RIPE NCC should not compete with the members, then that application is probably not eligible for service.

Migration and contingency planning so to speak, for those 75, 77 TLDs currently under service, these criteria will only be applied, that is the suggestion of the group, no earlier than two years from now, to avoid a sudden rush for those; they need to plan ahead because they had a reasonable expectation that this service would be provided for the time being.

And for documentation and transparency purposes, additions and removals would be documented, reported, probably in the usual report that the NCC /TKPEUFS here on some web page, details, again defer to the NCC.

And that is it, question, comments?

JIM REID: Any questions?

AUDIENCE SPEAKER: Stefan, AFNIC. A bit concerned about the letter actions. I mean the idea of doing a ping on the reply is a good one but you said that the details of the letter will be decided late we are discussion with the legal people. In my experience when you involve the legal people, it always end up being a lot of formality after a while so why not just saying that it's just a ping, a ping test, to see if there is some someone responsive on the other side and there is no need of special formality?

PETER KOCH: I would defer this to cave yea who is probably standing right behind you and fully trust the operations and technical people in the RIPE NCC to have a well?established relationship with the legal team.

AUDIENCE SPEAKER: First of all I wanted to thank you and people who were involved for the work you have done because this is I think a very healthy relationship between the operations we do and the Working Group and I really like to see this more for other operations as well. So second, to what you proposed we also really like lean documents in all that have so the idea of†?? I am thinking about right now and I think our legal is normally OK with that is lit weight MoU, half a page or one page, which basically acknowledged what Peter mentioned and if†?? maybe the criteria could be mentioned there but I think it would be in the level of MoU, I have to check but that is also what we do for anchor hosts and we are think about lightweight document most probably.

SPEAKER: Lars Netnod. You want something in there, at least to specify the responsibilities and the non?responsibilities that you have vis?a?vis each other, so some kind of exchange of intent or MoU is a good thing.

PETER KOCH: Yes, thank you. Since Kaveh yea mentioned it, before I hand the microphone back to Jim, I would like to thank Anand bud deaf from the RIPE NCC, Brett, Marco from D nick, as well as Jim and Jaap for this work. What you can see is the group was heavily biased towards people whose name begins with a J, but I hope that didn't influence the outcome.

JIM REID: Thank you, Peter. Just before we close this particular thing off, just to let you know what we think is going to be the next steps for this. The words which have been presented in the slides are going to be put into some piece of text, sent out to the Working Group list for further discussion and comment and if we are happy that†?? that that can represent what should be the way, the NCC should approach the continuation of this service, we will then have some mechanism for then writing that up. Now, that†?? there are essentially two potential mechanisms: One is we publish this as a RIPE document, ways very simple weight simple mechanism, we say this is the consensus view of the Working Group, bang we are done. Or if the Working Group is more minded to make this more formal shove it through the policy development processes and PDP. My personal preference would be to do it as a RIPE document, the Working Group will ultimately make a decision on once they see the final version of the text proposed and that will be happening hopefully sometime in the next couple of weeks. Is there anyone that would like to comment on the suggestions of the next steps or are they reasonably happy that is a sensible way forward. Silence implies consent. Thank you very much. So thank you very much, Peter.

And our next talk is from Michael Daly from Nominet. I got the name right and he is going to be telling us about the changes to the registry infrastructure at the TLD, thank you.

MICHAEL DALY: As Jim said, I am the I manage the admin team at Nominet, I am responsible for the core platform, storage, LAN, WAN and

Infrastructure transformation project which we ran over a couple of years. For those of you don't know, I guess you all do, Nominet is responsible for the dot U E TLD, we run various zones under that. Approximately 10 and a half million registrations at the moment. And we are about to have a few more under the gTLD programme. We have one of ICANN emergency back end emergency operators and we hope we will never have to run a brand under that.

We are public purpose. We are not for profit. We do generate some surpluses which we give to a charity. We invest and look after and so when we undertake projects like that we don't have to go out for external financing or anything like that.

We run the normal kind of stuff at registry, EPP, Whois, various websites and something called the DAC which is the Domain Availability Checker and allows our registrars to take a high†?? sort of massive query loads against the database to see what domains are available available or about to drop and that gives some of the load away from Whois for people trying to pick up valuable domains.

Autnum accepts very specific format from registered users and it it's a great way of making bulk changes of registrations to anything.

DNS light. So why do we need a transformation project? We had our own data centre in Oxford and very remote colow down in Kent. Like it was very rarely visited, not really tested enough at least not in a fully live scenario and not very sustainable in terms of expansion and development. We had, we still have, network presence in Telehouse, London where we have transit and peering, we 99% physical with lots of old hardware, mixture of Linux and Solaris.

What I am really trying to say is the environment was a bit like an old Land Rover. When it was brand new best of breed, fit for purpose, all the things you look for when trying to haul a couple of tonnes of kit up a mountain. It was getting old and difficult to repair and perhaps not up to the new role we needed it to play. With the business diversification agenda, gTLDs we are being about to ask it to do something it wasn't designed to do. What we really needed was a platform to launch a diversified business.

So, like any good project we started with some goals and requirements. We needed to rapidly deploy new servers and services and scale those up and down as we needed to. Rather than a primary and DR site, we wanted to be able to run services from any of our locations and not have lots of spare kit gathering dust just in case. We wanted to use any of it at any time as we saw fit. We wanted to automate provisioning change and configuration as far as pratically possible and monitor around services rather than servers.

We went out and picked two primary locations in the UK. DC 1 which is near London but not in it, DC 2, which is in London. The choice of location was driven by the need for good connectivity to the Internet, to the Oxford office and each other. There needed to be far enough apart so local events, power, flooding that kind of thing, couldn't or at least were very unlikely to affect both at the same time. Ultimately, we wanted to do needed fast connectivity for applications which over so they had to be relatively close. They are about 17 or 18 miles as the crow flies, that was about as far as we could get them.

So to be sure we could continue to provision DNS and customers could make changes to the zones if there was any sort of significant issues in in the south of UK we decided a third site would be a prudent thing to do. With the growing dependence of the Internet made us think actually further away in the UK wasn't sensible, go with offshore. So we have DC 3, and in a secret location, somewhere neutral, with mountains, where they speak French and German.

So, we have got our†?? done some due diligence, talk to suppliers and done lots of engineering and testing and this is what we delivered. 90% of the infrastructure is now virtualised†using VMware; choosing HP serves was quite an easy decision, we understood the tech knowledge and were comfort with what they were doing. Same with VMware, there was good pricing, pricing always helps.

Then we used to run EMC stands, and in terms of improved functionality the 3 par just outscored the EMC. Commercially, the 3 par made an awful lot of sense, it being an HP product again aggressive pricing, we are not for profit, public purpose, pricing is really important to us. We still do run some physical servers. Windows domain controls, oracle, oracle runs a main registry databases. We†?? it's not that we couldn't or wouldn't virtualise Oracle, it's just it was horrendously expensive, to put it in the virtualised was hundreds of thousands of pounds so it far outweighed the cost of 8 physical blades and some management time to manage it outside that have cluster. Switching came from our friends at HP. When I first started we had made that decision and it felt a little odd to me to come into an organisation that has the presence it does the on the Internet, not to be using Cisco we we got all the functionality we needed and we got a really good price because we took core infrastructure from HP, it's been a really good decision, it's been very reliable and done exactly what we needed to to do.

Firewalls came from Juniper, good pricing, knowledge and functionality. One of the requirements was platforms, protect ourselves from bug so we have Juniper and one and Cisco on the other, just to give us some things.

The core registry platforms runs on head hat, file sharing, e?mail etc., all runs on windows. So, we have two DC model, we run stretch VLANs across both sites. Low latency, it runs really quickly. Configuration in the network and storage are identically matched on both sites, same hardware, same configuration files, all identical. We use 3 par replication to keep all the storage together, and we use Oracle dataguard to keep database clusters up?to?date.

We can spend upserveers as we need to in the environment. We wanted to be able to move workloads around. So to do that we use VMware, in a management scenario four automated steps: Service shut it down down, ensures all replication is finished, and data matches and the server boots up. In the DR situation it just boots up the servers in the second data centre, up they go and off it goes. I will explain the problems with that in a minute.

The need to shut down a server before we move it means that if serves†?? services are based on a singler, we can't move without any downtime so the critical services, EPP and Whois we run at load balanced pairs and group them together into data stores. So, we move one data store, we checked severing happy and all working together and we are happy it's receiving track and we can move the other data store.

This does come with a bit of a caveat: If we have to move the Oracle database we need to few minutes of downtime to reverse the Oracle cluster, engineering that to run seamlessly didn't quite work so we accept there is a risk there and that is how we do it. Similarly, the firewalls and the load balancers, if we have to switch the primaries to second Reese then there is some connection resets that but we have done that in live running and there is no real problem, it happens and everybody reconnects and it's fine. We could automate all of those but we wanted to maintain the control and the ability to move when we wanted to. In a DR situation the serves just boot†?? glitch or something could just cause all the servers to boot. And if we boot boot servers with the name name and IP address and MAC address on the same network that can cause a lot of confusion and some issues. So, somebody always has to hit the button. We have to be sure this is what we want to do and somebody has to press that button. So there is problems we hit the button off it goes, we know what we are doing.

DC 3 we keep in its own little bubble, we don't transfer in that sort of environment and we shift the backups all go over there, Oracle keeps everything in sync so we run independent servers there that we can configure. We flick DNS and can be running from DC 3.

So, all this great new stuff. How do we monitor it? I said we want it to be able to monitor services as a whole rather than specific servers. And of course looking for that /TPAEUBLD single pane of glass into the infrastructure. We didn't find a magic bullet so went with a /HAOEUB /PWREUD /SPWHRAOUGS. There was some interesting products in the market. We use V fabric I per I can and write custom applications that that which monitor our applications at that core level so we can see how many users are connected to EPP and how many pool connections are using the database, that you will kind of interesting stuff that we need to monitor.

We use HP insight control and IMC to look after the hardware and the networks and when they with us V C /KO*PS to manage the vitalization environment, /KWOPLS a rather handy custom interface which can pull data from lots of source, we use it. This is what the†?? this is what a typical V C Ops dash look looks like. This was last Thursday so our entire world is VMware had a health score of /# 9, health score of 0. And then we can drill down a bit more and you can see individual elements, so the yellow and orange boxes you see towards the bottom will be things that maybe this had a few anomalies and we needed to look at. As I say, nice big dash /PWAORBGSDZ we look at this, it helps us out.

As well as alerting on hard thrash†?? it also figures things out for itself so it looks at trends and decides what is normal and isn't. So, for example, we know that EPP gets a huge spike from one of our registrars on a Wednesday evening. VC Ops has learned about this so it doesn't wake anybody up. If there is a problem with Whois, it will wake somebody up so we can have a look at it.

Just a quick alerting flow. Everything flows up, feed up into VC Ops custom, it feeds into service now and that delivering emails and alerts based on what is going on. Based on the server that is alerting and the services it's aligned to and priority level we give it it decides what to do. So an EPP server with 80% disc space, the person look at. EPP hits 90%, somebody gets woken up. Very simple.

So, we went from a Land Rover, we now have a spaceship. It cost us 4 million pounds. We delivered on time and on budget. I think the one thing I need to say, I can't thank my team enough for delivering this. They worked really hard in what were trying circumstances. They had to break in a new manager within weeks of the project starting and we couldn't do it without them. Some of them might be watching, you did a stunning job, all the guys in development. Well done. That is it. Thank you very much.

JIM REID: Are there any questions, comments? No. You are free. Thank you. Who is next? Yes StÈphane Bortzmeyer.

ST…PHANE BORTZMEYER: I am going to talk about small security problem that you may have observed a few weeks ago, and which involved Turkey. As you probably know, there have been awful mining accident in Turkey yesterday and hundreds of coal miners died in an accident so this is to remind us that whatever the security problems of the Internet, there are things that are much more dangerous than surfing on the web.

So, back to DNS. Context is some people were criticising the government, most governments don't like to be criticised, because it's 2014 people don't express criticism only at the bar but also on Twitter, YouTube, things like that. So the government requested the Internet access providers to shut down the service, through the use of lying DNS with reverse, and you get instead of IP address of a website with a nice message saying you should not do this or that. That was on march 21st. That is very common technique, it has been used in many other countries, for instance, I am French, and in France it's used by the quote in the case of streaming which was a site of video distribution, the†?? has been sentenced†?? the Internet access providers have been sentenced to use the DNS to filter out requests for other streaming. Same thing for on?line Gambling sites. So this is not specific to Turkey, it's quite common.

But then what happened? People decided to walk around and to use, for instance, a resolver which was not lying, Google Publin DNS, we talked about this service this morning, so I assume everybody knows about it now. So we saw people on the streets of Istanbul using the walls to advertise the IP address of Google Public DNS which, in my opinion, is the first time there have been people using the walls to display IP address instead of anti?government slogans.

So, people were told to modify their DNS configuration. By the way, on typical operating system changing your DNS resolver requires more instruction than what you can put on the wall. Objecting. But it's a statement. So people were using public Google and getting their correct address for Twitter on YouTube. And the government decided to block to filter out any request to 888 or things like that. That was four days after on 21th of March.

Then, the problem is that many people were using this open public resolvers not only Google public but also things like open DNS and things like that. So many people complained because the Internet was no longer working at all.

So, the last step was on 29th March, to inject false roots for important public resolvers, for example there was a route for which was sent in the routing table in Turkey, redirecting people to an actual DNS servers so it was not a black hole, there was something at which was talking DNS and answering requests but of course answering with lies about and which were redirected to this IP address which belongs to Turk telecom which is biggest access providers in Turkey.

In my opinion it's a new step, through the DNS as I said is used in many places in my own country, for instance, but in many others but in this case it's no longer simple sensor ship, it's actual hijacking, in my opinion, answering IP address that is do not belong to you in order to fool the users and send them elsewhere. So it's a very unfortunate and very new step in Internet censorship.

So after that, apparently on the 4th of April, after a lot of protest by people, remember that Turkey is the host of the next Internet governance forum, the forum of the United Nations which talks about Internet governance, freedom of speech and other interesting things. So one week after the Turkey government stopped lying on this resolver, on a few days after the routes were no longer announced. I didn't check today what happens, I have been told that two days I was in an important meeting between people in Turkey about what is going on and the about the Internet, I don't know what will be the outcome, but the problem that I am talking about lasted only a few days. Still, it's may start again at any moment.

So, some people will tell me, OK, you serve up the hijacking routes, what proofs do you have? Many. Unfortunately, Google public DNS does not provide even very simple debugging tools like NSID or standard bug, it could have been useful, for instance, two years ago, the RIPE NCC did a survey of the NSID answers to request to K?root and it appeared there are some strange machines which replies to the IP address of K?root with an NSID which is not an actual NSID of any of RIPE NCC instance. So very simple tools can be useful to detect the this sort of thing. Of course they are not real security measures because someone who hijacks roots can also add false ID. So the real proof that we have, first Turk telecom has a looking glass which is very convenient. You can see what is going on, what they do, and you can see that there was a /32 which was injected in the I GE P, in the interior routing protocol. Then we had the DNS request from RIPE at at loss probes. You know there are two kind of people who do measurements, do you use the†?? and Geoff Huston. In my case, I ask the RIPE Atlas probes to resolve names like in the RIPE Atlas API you can select the probes by AS numbers, prefix or /KUPB /TRAOERBGS and there are ten probes currently running in Turkey so all these probes but two were reporting the IP address; the two probes with the correct answer were probes which were connected to a network which was completely tunnelled to end point somewhere in Germany in order to work around censorship, which, by the way, that /TR*EPB work. There have been not so many DNS requests by human users in Turkey. In Turkey there are many people knowledgeable about the Internet and DNS so many people were doing DIG or DNS lookup in many places. I tend to be careful with trace routes when it comes to security, because for an operator, network operator, it's too simple to mess with trace route, to change things, to intercept the ICMP packet going back so trace roots are not really proof. Late ensee, it's more difficult to change, it's possible to increase it but to decrease it it's typically not possible unless you happen to find a way to increase the speed of light which is out of scope for this Working Group.

RIPE Atlas probes and /REPB cease probes, the measurement was not done by me but by the RIPE NCC people, to show that the latency suddenly decreased. Of. It was typically 70 milliseconds to go to 888 and suddenly it was down to 10 milliseconds. Since Google has no machines in Turkey it could have been done only if the machine was suddenly much closer from the user.

So I believe that there is absolutely no doubt of what happened, even if there was†?? as far as I know, there was no official communication either from the Turkish government or from Turk Telecom about this hijacking. It was done but done silently.

So, what of†?? of course, we are already, all of you, thinking about what could have been done to avoid the problem. So that is it.

DNSSEC, well DNSSEC could allow to see that the answer was†?? BUG S1 was a lie, if†?? and too big if ?? if,, etc., are not signed so it's a first step to which before we can use DNSSEC to detect censorship and the second problem is that validation needs to be done properly which means on the user's machine because Google public DNS validates with DNSSEC as you have seen in Geoff Huston talk this morning. But, between Google public DNS and the user there is a famous last mile, and in the case of Google public DNS last mile is many miles. And during this last mile many things can happen: People can add or delete it or do whatever they want. If we want to use DNSSEC in hostile environment, we need the validation to be done on the user machine.

Still, DNSSEC would detect the problem here, but of course, you still wouldn't have access to Twitter and YouTube so when the goal is not to redirect people to something else but when the goal is to shut down the service, DNSSEC is not really a solution, with DNSSEC at least you know that there is something fishy, but you still cannot access your favourite service.

Authentication of the resolver could have been a good idea, at least to detect that the 888 was not the right one. As far as I know, but Google people may give details, Google public DNS has nothing to /A*UT enterify the resolver. Open DNS, which was also hijacked in exactly the same way, has something called DNS script which is based on DNS curve. I don't know how many open†?? I don't know how many open DNS users are actually using DNS script. I have seen no report from Turk bee it so I don't know if it was really efficient but if you use it you can detect that there is a problem that you are not talking to your favourite resolver. This is a big hole in the DNS security, even if you have DNSSEC, if a link between you and your resolver is not safe, then DNSSEC does not help a lot. So, in my opinion, that is something that really we should address, try to find a way to identify the resolver. It relies on and of course if a secret is shared by all the users of Google public DNS, it's no longer a secret. So we need something better, such as, I don't know, IPsec 60, etc.

What about RPKI? Because this is a RIPE meeting, many people will say that is simple, we just have to deploy RPKI. Indeed, there have been an article on the website of the Internet society about how hijacking of Google public DNS in Turkey should be†?? should remind us that we have to sign with RPKI, we have to validate the root, etc.. in my opinion, it's wrong. It's wrong because RPKI defects the BGP announce the false BGP that you receive from the outside, but when you inject yourself in your own routing system, in your own EBGP†?? IGP, when you inject a false root, RPKI can do anything. I mean, it's a network of Turk telecom, they can do whatever they want with it. Even if the routes to Twitter were signed with RPKI it wouldn't pre vanity local policy to override them with anything they want. So in my opinion, it has nothing to do with RPKI and RPKI is useful but not in this case.

On the last step, as I mentioned about DNSSEC, it's detecting there is a lie is one thing but working around is another problem because the attacker wants to redirect to you somewhere else, for instance, for phishing purpose, it's good to be able to detect the problem and to block it. It's the goal of the attacker is to prevent you to go somewhere, then authentication is not simple solution because you still don't have access.

So, what was possible during the crisis in this specific case, in the case of Turkey during the crisis, it was possible to use another public resolver, many were blocked, Google public DNS, open DNS, the thing at the Commodore, and thing like that but some were still open, Telecomix was not hijacked probably because the sense source didn't think of it. Run local with new machine with DNS validation etc. Because port 53 was not blocked, so it would have been possible to run your local DNS resolver, of course it's as always with security, it's arm rest situation so if people had started to use local DNS resolver the next step would have been to block port a 53. The best solution is to use tunnels to all this sort of thing. For instance, this is what happened to two of the RIPE Atlas probes in Turkey, they were connected to a network where the entire traffic was tunnelled to the outside before being distributed to the Internet so in that case it was a good way to work around the censorship, and until the moment where all access is blocked etc.. there is no proper technical solution to this sort of problem. We can use this techniques today to evade most ordinary censorship attempts, but still it's mostly political problem maybe also legal problem because in my opinion, announcing routes that do not belong to you is a criminal offence, or should be. I don't know how it is in practice and I am not sure that it was a solution.

So, some reference because many articles have been published about it. For instance, a document by AFNIC scientific /SOUPBL here was written before the problem in Turkey and is a study study of censorship with DNS, how to do it and not to do it and why it's a bad idea and what are the consequences.

Thank you for your attention. And†?? because this is a meeting of engineers, I expect that many people will have great ideas to fight against censorship but remember that the best one is probably to be involved in public life because this is a place where things really change. Thank you.

JIM REID: Thank you, questions and comments?


ROBERT KISTELEKI: One of the mentioned references is a pointer to a labs article which was written by†Emile, and it has a nice illustration of what you explained, how the latency dropped from the probes to the theoretically the same address, the, so just pick it up.

ST…PHANE BORTZMEYER: I recommended you read it.

ROBERT KISTELEKI: Thank you for putting it up.

AUDIENCE SPEAKER: Edward, thank you very much for a brilliant presentation, it was very interesting, thank you.

Well, when you are talking about this crime type of thing about injecting false route into routing table of a country, I don't see very much difference when you just facilitate DNS name. So it's completely different techniques but it's actually the same actions, so you redirect the user to false target, doesn't matter by DNS or by BGP, it doesn't matter. And as you explained, there are many, many techniques to technically try to avoid it but there is no technical way to resolve the issue on that level. So, there should be found some political maybe measures to fight against this type of crime which is done by a government. So maybe there is something that need to be decided for all the countries who are involved in this and so maybe on ISP level, maybe not on governmental level, so if an ISP goes and does what the government requests, then the ISP must be, somehow, regulated. Thank you.

WARREN: Google. I realise that this is far from a perfect answer, but further back you had something trying to detect if you are talking to a Google node. It turns out if you do a look for O ? O.mydata ?? a text record, you get back a text record that is your address. I don't think that that was working when the Turkish hijack was happening. It's not NSAD but something that is a little trickier to forge.

ST…PHANE BORTZMEYER: I didn't know about this thing. That is interesting. We could use, for instance, a RIPE Atlas probes to query for this name to see what happens.

WARREN: Another ?

ST…PHANE BORTZMEYER: Where was it documented by the way?

Warren: Somewhere on?line.

JIM REID: Google for it.

AUDIENCE SPEAKER: It's not on the Google site, it's something that somebody discovered, so I don't know how this scales.

Another interesting censorship thing that somebody in the Turkish government was that they simply blur out offensive tweets. How you do this for an ascii protocol I haven't yet figured out, but it was interesting.


CARSTEN SHEIFNER: I just wonder whether you or anybody else has received or seen any, say, official or semi?official answers or reactions to all the reports and in particular your report as well, the thing you have written up about it, whether there has been any kind of thing as been published as an answer, yeah?

ST…PHANE BORTZMEYER: For myself, I believe that my article was the first one on?line and I received many feedback from people in Turkey, but not the government or people working in Turk telecom, ordinary people and users. For the other articles, which were often published by better known organisation, I don't know did the RIPE NCC receive anything after the article on RIPE Labs? I don't think so, I have never heard of it. The author is there. So you can ask him. But no, there have been to official reply. Remember it was not announced to the peep who are most concerned, the /T*URPB Turkish people themselves. There has been a lot of boasting from the prime minister about how he will not bend in the face of pressure from other countries or his own people by the way. His slogan for the election was "I won will"†?? so he does not seem to be the sort of guy who wants to engage in some bottom up†?? I may be wrong.

AUDIENCE SPEAKER: It could even have been that Turkish Telecom would have issued like a press statement that they have done engineering test cases or whatever, this kind of like awkward explanations†??

ST…PHANE BORTZMEYER: We are the new employee and he made a mistake.

AUDIENCE SPEAKER: Something like that, yes, not even that as I understand. OK.

JIM REID: It makes a change to my dog ate my homework.

AUDIENCE SPEAKER: Blake with L33. So to paraphrase some of my libertarian friends, every time someone says "there should be a law an engineer cries". I think that it's a very slippery slope to say that maybe this is a crime or something when it's not actually perhaps illegal and maybe opening the door to invite more regulation into our industry whereas we are sort of trying to go in the opposite direction. Especially in the last few years when there has been very large amounts of push towards this. Speaking personally, in one of my clients, I administer, for example, a network that uses a lot of captive portal stuff, specifically on hotel wi?fi, that we have had actually had to inject DNS server addresses into our own network, simply because the network historically was using someone else's and they got renumbered and we still had 4,000 CPEs giving these addresses out etc. So they got redirected†?? we weren't doing anything malicious with that, we weren't falsifying any DNS queries, but nonetheless it's a very slippery slope to say we can't do this but maybe there are legitimate technical uses for the same thing and it's not necessarily the technology or the technique that was the problem but the intent, in fact.

OLAF KOLKMAN: NLnet Labs. See part 2 I would almost say, in the collaboration Working Group this week I will be presenting on behalf of somebody else, but also something that I have been working on in the context of the IAB, an analysis of if you have to do filtering because of legal reasons, technology doesn't know about legalities, I mean packets are packets are packets. But if you want to do filtering, what is the best place to do that? And what are the effects of filtering, for instance, the DNS and might be unintended effects of that? So, basically, a heads up for part two, that would be more targeted towards the audience in the Cooperation Working Group, so trying to stay away as much as possible from the actual technology. So a little bit of self advertisement, I guess.

JIM REID: A question from the Jabber room.

EMILE ABEN: Yes, there is a question†?? somebody that want to let us know tomorrow during the coop Working Group an anti?sensorship technique will be presented,†?? he is the author of a document that will be presented tomorrow. And I have got another question but I am not sure, I didn't get confirmation that this should be read to the whole audience.

JIM REID: OK. No more comments, we will just say thank you very much. Great presentation.

Next up we have Robert Kisteleki from the NCC who is going to give us an update about planned changes and visions to the NCC's DNSMON service and following that we will have our panel session with Robert, Michael Daly and to discuss various other aspects about common approaches or thoughts about DNS monitoring and more standard or uniform way across the industry, maybe, perhaps, possibly.

ROBERT KISTELEKI: I work for the RIPE NCC in the R&D department and I would like to present you what we did and still plan to do regarding DNSMON.

For those of you who don't know, I don't know if there is anyone like that in the room, DNSMON is a service that the NCC has been providing since 2003; it is monitoring so?called important DNS zones, not everything. Whereas the definition of "important" is either it has to be a root zone or a route server, it has to be a classic gTLD, so now they are proliferating but back in the day not so money and subset of cc TLDs, mostly in this region. Also infrastructures like ENUM and so on. It is a combination of data gathering and data visualisation, it uses RRDs back and forth to show all the interesting bits and pieces and it is based on the TTM, the test traffic measurement, network that the RIPE NCC built back in the day to actually collect the data.

As you have probably heard, we had to work on renewing this whole infrastructure, for a couple of reasons. One of them is that the TTM infrastructure as such is ageing itself, it has been out there for ten, twelve years or so. The back end itself was difficult to maintain, it's built on FreeBSD 4 or 3 or minus something, so very, very old version. So it would have required also some investment to actually keep it alive. You also have heard a couple of times during this week that the NCC active measurement network that we are actually maintaining and developing is RIPE Atlas. Atlas has already been capable of doing DNS measurements, so that is one, and that is good. We have more and more RIPE Atlas anchors so we have in addition to the small, less reliable stable vantage points with high bandwidth connectivity. So RIPE Atlas seems to be the right framework to reimplement DNSMON in.

There are a couple of differences in the new DNSMON and the old one. Obviously, the vantage points are different because they are no longer TTM but RIPE Atlas. There is a huge overlap between where we did have TTM boxes and still have some of them, and where we have the Atlas anchors but it's not an absolutely matching set.

The data, if you want to get to the data, if you did get to the data before, that changes as well, because RIPE Atlas is spitting out Jason structures instead of flat text. But with the change we wanted to introduce something new and better if we could. One of these things is the support for TCP queries which has been lacking in the previous DNSMON implementation. Also, since the RIPE Atlas network is capable of doing trace routes from the same vantage points to whatever destinations, this was a good time to introduce those as well. So any time we are measuring a particular DNS server from the Atlas anchor vantage trace routes, with varying frequencies, it's not contactually matching up with the query but if you need to check what happened, why did my packets suddenly take this much time there to get there or not get there, where they blocked, trace route is your friend and we provide that data already.

There are a few bits and piece that is we changed, sometimes we had to change, sometimes we thought it's a good idea to change and most of the time we discussed it with the involved people. One is, because the visualisation is no longer server side, we are not generating images use RRD tool but instead giving Jason structures to the browser and there is an interactive tool that you can play around with, zoom in, zoom out for change the servers and so on.

We did not provide or from now on we will not provide the RRD.

The measurements that DNSMON does will not be retried on failure. This has been discussed with the early testers and we asked the question: If we are doing this again, how should we do it and the majority of the responses were do not retry on failure, we would prefer to see errors as soon as they appear instead of masking away, and everyone is happy although half of the packets were lost.

Also, we are trying not to do the visualisation delay, in the old DNSMON there was a distinction between who is a DNSMON user and who is not. DNSMON users had kind of a preview, it was a semi?realtime data feed, if you will. The non?DNSMON customers had a two?hour or three?hour delay to the visualisations, we are trying to do away without that in the new system, partially because the measurements themselves are public measurements done by RIPE Atlas so the data is out there anyway.

These are the measurements that the new system does. Notice that we are doing now the SOA queries over UDP and TCP and NSID deflect there just to get the instant information, which is also used if you dig yourself into the visualisation and you say what was that and then you get more information about that including NSID.

And those two are used to do the actual visualisation of zones and servers and probes. We also do a relatively high frequently host name†?? relatively low frequency version of BIND which could be used to determine instances, visualise instances which was not a feature of the old system, it could be a feature in the new system if you guys say that should be the case.

And also, the trace routes are there, so each and every server from each and every monitoring station is trace routed, too, so we can show that data. We do this over IPv6 as well as the legacy protocol.

So, time?line:

The development for the new system was done end of last year, beginning of this year. We did some internal test in January and invited 25ish people who previously showed interest in this migration to participate and test the system early on and based on their feedback we made changes here and there.

Every since early April the system has been in public Beta, if you go to the service now we will see the well?known Beta logo there. We plan to have to take it out of the public Beta status in June and we will declare it to be a production service. Obviously we will still tweak it if there is a need, change this and that but we are, for those things we are really going to look for your input, what are needed and not needed.

After that, in order to help people migrate away from the old system to the new system if they wish to do that the two systems will run parallel to the end of June. Since the message about changing has been out for a couple of months now this should be enough but if it isn't please say so: Beyond that, we will stop the data collection in the old system somewhere midyear, the plan send of June, beginning of July so that the TTM hosts can also stop their TTM machines, which some of them will be very happy about because they can reuse the machines as GPS based MTP clocks and so on. At the moment we plan to stop this data visualisation in the old DNSMON by the end of the year. If there is a need to revisit that, let us know. This is the plan at the moment. And we do plan to keep the data from both the old system and the new system around for as long as we can. There is no reason to delete data if we don't have to.

Just flashing up, this is†?? I will not go into dynamics and how it works, explore it yourself. One thing I want to highlight is this guy, which is a pair of thresholds that you can set yourself about how sensitive the system should be when it visualises data to you. So you can say, I don't care if packet loss is smaller than 50%, it's up to you, then you can set it to 50% and then everything above that will be coloured based on your settings. This let's you do a couple of interesting things, but also if you play around with your mouse you can see that you can zoom in, zoom out, both in time as in the Y axis which in this case is servers but if you click on server you get probes view and click on that and even more details.

Now, I pretend that I clicked on one of these links, then I get a somewhat different, the basic idea is the same, shows the Atlas anchors on the left?hand side and just as before horizontal red lines are bad, vertical red lines are worse.

And finally, if you dig yourself enough, deep enough, then you will see this. This is how a particular vantage point saw all the serves that it's monitoring, so what you can determine from this is if you feel that this vantage point cannot get to you, something looks wrong you can check if it's only you or that particular vantage point cannot see anything at all N particular, this shows that one of the PL servers is unreachable from this vantage point over IPv4 UDP but IPv4 TCP, IPv6 works well. So switch to IPv6.

OK. Having said that, gameification, it seems to be the word nowadays so we are going to try to apply that to DNSMON and I know there is a risk of our servers melting down in the next hour. Still we are going to this. We are going to play a game named capture the flag. The idea is that you should play around with the new DNSMON and find something that looks like a flag. The more it resembles the flag, the better. The higher the resolution the about thor. If it does not look like a flag, if it looks cool, submit it and we will pick a winner by absolutely subjective measures and we will announce that on Friday, so I think, yes, we would appreciate if you submitted your Permalink, which is just URL that is up there, by Friday morning. So we have a bit of time to actually look at the things that you submitted.

As you can see the rules include some saying†?? let's just call them rules, please, please don't fiddle with your zone, just to make it red. Please don't fiddle with anyone else's zone. And we would also appreciate if you didn't fiddle with Atlas itself. Don't DOS the system to get a yellow colour.

This is the reason why I showed that particular setting to you because if you play around with that, you can actually achieve white, green, yellow, orange, red and a couple of things. If you can do the Union Jack, awesome. That will be the winner, so...

JIM REID: Do it while there is still a Union Jack.

ROBERT KISTELEKI: Be creative but stay within the rules. This is the URL that has been mentioned before, just to make it explicit and are there any questions?

PETER KOCH: DENIC, no hats. First of all, thanks for updating this, I guess this new DNSMON is a real improvement and it was entertaining to participate in the Beta tests at the moment that Robert twisted my arm to it. Thank you. Now Robert can also read my mind. There is one feature and I have two remarks, one is this: There is one feature that was when the old DNSMON was started, was explicitly mentioned as anti?DDoS control feature, that was the two?hour delay window, and was, as you said, kind of the preview for members because they were trusted and countable, kind of, and maybe accountable. I don't really buy the argument that delaying this doesn't make sense because the data is there anyway. It's true that the data is there anyway, and people could do their measurements anyway, but that is similar to would you please hand me your purse because I could twist your arm to it anyway. And the second part of it is you are not only delivering the data, you are also delivering the publication and monitoring channel which is like not only allowing me to get you to steal your purse, you also make it happen in headline news 8:00 on television. And that is the crucial part. So I would really, really encourage you to revisit that decision as one of the customers for this.

That said, there is the other part, I guess it was slide 6 or a bit later talking about the decommissioning. So, thanks for giving us time to change our own software where we collect the data and so on and so forth. I guess from our perspective, we can live with that. From kind of the armchair historian perspective, it's a bit sad that the visualisation is going to go away at some point in time. Maybe there is a way to keep that open, somehow. There are a couple of old URLs that might break where people have used this in maybe even RIPE Labs articles, and for some research that is going to be done in ages from now, having the vehicle there would be great.

ROBERT KISTELEKI: Yes, there is a fair point. And what we could do, which could be a middle ground somewhere here, is to take a look at the actual usage of the old DNSMON visualisation, somewhere towards the end of the year, and make a judgement call based on that. If there are two users coming to us once a month, that is probably not worth it. But that is something we can do. We can also explore what it would mean to keep it alive indefinitely. I think that our G I I department would say that OK we definitely we can do that but then it requires some resources to keep it up and maintain and we are using years' old version so there is something to be done about that as well.

Jim Martin: ISC. During the Beta testing of this you guys found some issues with the ISC infrastructure and I just wanted to say some of the advantages to this new system have really uncovered some issues, and I am encouraging everyone to take a look at it again because it can actually be really useful and thank you for that.

DANIEL KARRENBERG: RIPE NCC. One remark: You don't want to burden the NCC guys with running that visualisation indefinitely. I wrote it, I know how readable it is. So†?? and I think, really, if†?? as long as the raw data is there, one can definitely make a little thing that if somebody calls up some of this URLs, these old URLs, gets a message saying sorry, no visualisation but the data is over here and I would think that would be enough. But the thing I really wanted to say, and ask, is, you know, Peter just said, OK, I want my delay back, and that is†?? and that is obviously easy to do, you just delay it, and that would only go for the visualisation because it's not easy to do to hide the underlying Atlas measurements because that would be major surgery to Atlas, which would cost a lot. But if we just assume that we want the delay back for the visualisations, then my question is: What should be the criteria to get the non?delayed version? Or are we all happy with three hours' delay? If we are then the discussion is moot. If we are not, then we have to determine the group of people, people behind screens that get access to the undelayed version and it needs to be a group that we can easily authenticate. So if there is really†?? if Peter's point of view has some traction in this group, then I would encourage this group to think about those questions.

ROBERT KISTELEKI: It's a very valid point. One addition to that is that DNSMON officially became a membership service so it's now more difficult to make a distinction between who is DNSMON user and who is not, and if the membership is, including old users then we will open this non?delayed thing up to 20, 30,000 people right away, which basically means everyone.

JIM REID: Can I make a suggestion here: Could you perhaps put something on the Working Group mailing list outlining the potential advantages and disadvantages of having this delay or having it not†?? non?delayed, both points of view of the management of the software and the development of the system and the ongoing maintenance potentially of legacy infrastructure that you want to put a bullet through the head of

ROBERT KISTELEKI: We can do that.

AUDIENCE SPEAKER: I have a comment from Jill mass en†?? he agrees with Peter comment on delay in the visualisation and he wants also to answer to Daniel the delay for everyone would be fine, go to raw data if want it.


AUDIENCE SPEAKER: Retracing the three points that was brought up, for the serve my suggestion to the Working Group if everybody agrees is to actually, as Robert mentioned, revisit it next RIPE meeting, number of visits we had to the old DNSMON and if we see there is real traction to that we can obviously keep the server on it, more level but try to keep it for longer period, if the Working Group is OK with that.

JIM REID: Certainly, thank you Kaveh. I think one of the things that would help here, we have some more context from the NCC about some of these infrastructure issues, we have to be very careful we don't start intruding into micro management, that is not the business of the Working Group. We want some comments about the functionality and what kind of features we would like to see in the software, whether you use a particular tool or platform that is nobody's business but the guys having to provide the service.

AUDIENCE SPEAKER: Thank you for clarifying that. Second point is basically about the planning and the time?line we had, which was†?? so, I also sent an e?mail to the Working Group, and basically, this is what we published and we ask for feedback. There is some inter twist between this and the TTM, we want to let them know well in advance that now we won't use your services for DNSMON so you can shut down your servers and all of that. Because of that, we would really like to get the input from the Working Group as you know we didn't get anything†?? we didn't hear anything. We had some private e?mails but nothing on the list, I would like to get direction on that.

JIM REID: I would just reinforce that message from Kaveh to the Working Group. If you have any comments on the time?line suggested posted on the list recently, please make it known because at the moment there hasn't been much discussion about it and cave†?? do we still this is still an open issue that might need some kind of positive there to say yes or no. Please speak your comments on the mailing list and soon. Thank you. Thank you very much, Robert.


Now our last item of business business for today is the short panel session to discuss various aspects to do with DNS monitoring and we thought it would be a good idea to try and get the people who have got some thoughts and suggestions for new or different DNS monitoring services to talk about what they plan to do. And some dialogue. Could I ask you all to come forward.

JIM REID: Anyway. There is a couple of things that have been going on recently and Lars has got some notes here that the route server operators are trying to develop something here, I know Michael has some ideas about of Nominet are doing. First over to Lars?Johan.

LARS?JOHAN LIMAN: I am with Netnod in Stockholm, we operate one of the route name servers and also commercial servers for TLDs. What we see is an increase, a proliferation in the various types of statistics that various groups of people want to have. We have internal statistics that the route server operators need themselves. There are public requests for statistics from route server. There are various customers that want to look at various types of statistics and there is also limitations in what the software can provide. So there is no common agreement about what statistics to collect or how to form that when we exchange it with each other and also what the various†?? variables, if you wish, what they should contain, what the counters are and what they mean and the context of that.

So we started to talk about that in Netnod. It turns out that ISC has been thinking about that as well and as soon as we mentioned it to other people it turns out that ideas have been floating around about this so we have†?? we mentioned it during the meeting this past weekend, DNS operations and analysis research centre wanted to help us out so they have now created this mailing list and a WIKI page for this that you can see behind me. This came up I think this morning or yesterday evening, so this is kind of very, very brand new but we have an idea that we want to start a discussion regarding this and see if we can find some common things that we can agree on.

One basis for this is a document that is being produced within the route server system advisory committee to the ICANN board of directors, and this document specifies a very, very basic set of counters and variables that route server operators will be expected to report publically and we can†?? we realise there are more data so we are also looking at kind of finding a small common list of things to report, so?and?so people can compare statistics from various sources.

JIM REID: I have one small question of clarification on this OK sponsored list, is this open to everybody or just for OARC members or just to be decided.

LARS?JOHAN LIMAN: It's open to everybody so we are looking for input here, from people who use this information. In my profession it's mostly TLD operators but we are looking for other consumers of the data, we are looking for software providers who are looking for what to implement in the code and we are also looking for operators who have to deal with providing this information to customers and to the general public.

JIM REID: OK. Thank you. Michael.

MICHAEL DALY: Thank you. So statistics and interesting data about about what is going on in the zones is great and handy for us to see, from an operational perspective we would like to see proper monitoring, gTLDs, how is their reachability looking from the outside world and the ability to pull that data out in API or have it alert us that something is wrong. We are our own probes out in the wild that do this sort of monitoring but having it in an infrastructure that isn't managed by us is quite nice because we can always control our own /TPR*EUBG infrastructure and tweak it and understand where it's coming from, somebody that says it's up and down and not quite right would be really handy. For it to be Anycast aware would be quite handy as well, especially as we move into delivering that infrastructure ourselves.

JIM REID: I have a question for you, Michael. On the ICANN side of this because you are now involved or Nominet involved with gTLDs, this backup provision, are ICANN segment any requirements from you from reporting and monitoring point of view.

MICHAEL DALY: We do have some requirements in our we monitor the domains under our management so we can say this is up and down and how we get alerted. They are not particularly specific right now. They are going to get more spec and more developed. There is a lot of speed of transition that is required, if an' vent is declared, serve it and monitor it in four hours so†?? tend to be speed requirements rather than specific data.

JIM REID: Thank you.

ROBERT KISTELEKI: I am a recovering Atlas addict. I am not recovering, actually. So one thing that we can adhere is some kind of independent monitoring and independent measurement of how a particular DNS zone or server works. Every DNS operator that has†?? operates whatever kind of DNS zone, has some kind of monitoring for their own network but such as Atlas can give an external view. We have many of the components that we need in order to provide this as a service that is as easy to use as possible for the users. The RIPE Atlas can do DNS measurements, we have a wide set of probes all over the place. Recently, I think in February we introduced the so?called status checks so if you have a running measurement you can just go to Atlas and say is it OK for your definition of OK, you have knobs and filters that you can play around with and the system tells you yes or no. At the moment that is only applicable to ping measurement because for that the semantics were really understood. If we want to do such a thing for DNS measurements then we need expert within on what that means. You want to compare server errors to no errors, server fails and all kind of things. From our point of view the providing external monitoring to these kind of zones and giving you easy to digest data is definitely doable, we just need to understand what is the need.

AUDIENCE SPEAKER: Keith. Really just two brief comments, one is to restate OARC support for this initiative. It's pretty core to what we are doing. Another way you can monitor TLDs is that OARC has a service which is kind of languished over the past or year or two called TLD MON, it looks at all the top level domains. We have revamped that over the past couple of months, so if you have a TLD operator it's worth a look, it gives you another slice of the pie in terms of visualisation.

JIM REID: Thanks. I have another question for the panel here. We have been talking about monitoring in intensive context, firing packets and looking at what goes back and guarding statistics but what about the actual internal monitoring actually in which the name server software is itself performing, have you given any thoughts what about should be done there or is that something concerns you. It may not matter for Robert but to some of the guys in the DNS team in the NCC but Lars and Robert, what could be done there?

LARS?JOHAN LIMAN: What†?? one I think that we are looking for is to have this common set of counters and a common understanding of what the counters means because we don't want to depend on one single type of software so we want too far spread of different types of software in our cluster of machines. But if we cannot compare the statistics it gets very difficult and we have to create all kinds of translation mechanisms or we have to look at the packet stream to try to figure out statistics that are actually better figured out inside the server and also if we are going in a way that has been suggested to encrypt DNS traffic looking at the packets is not going to help us very much when we look for statistics. So to find common sets of data and more importantly, common form it's a for changing the data, that is the real focus I have right now on this part. But that said, the traffic provides a different set of information that you can look on externally and you can do it both by doing what Atlas doing sending queries and looking at responses but you can also look at servers on the incoming traffic and filter out and look at the properties of the flow of incoming statistics. And I would say that I would also like to be able to provide that type of information in a standardised way, so that you can create various types of systems that look at this. You can do it either by having internal counters in the same nerver, you can do it by having the name server provide a stream going out to separate monitor or packet collector in front that does its own monitoring, self?sustained entirely but exchanging this information after the fact that the really important thing. And also so that we can do in a way so we don't have to funnel everything back to a central system, at that calculations and analysis can be done at the local site. Because if you operate large cluster systems, sending too many data back home is a tremendous load on the network that is very difficult to handle, not necessarily by the company that handled it, but by the Internet between the sites because it's not 10 gigabit pipes everywhere.

MICHAEL DALY: Number of /TPHA*Gy checks that†?? whether services are up and down and we use various external methods to probe the name servers for answer†?? as we refresh, that will become hyperic and that will do a bit of a blend, there will be a separate monitoring server in each node that will sit and pull those statistics and understand and there will be hyper I can running on the name servers themselves and we will build a set of extent probes that will come across the Internet and pick up to test the zone and validated properly and is the TTL boot right and does it get the right answer.

AUDIENCE SPEAKER: NLnet Labs, software developer, so I will be worrying about whether there is a set of requirements how to implement that. You already hinted something for what I was going to ask on the OARC meeting there was a presentation on DNS Tap which provides a way of†?? from the name server getting things like query logging, collecting the data. There is a defined representation I was wondering if that representation could be useful to as a basis here, which is the proto BoF thingy. Do you know about that? And if so do you think that could be useful as a basis? And if not, why not?

LARS?JOHAN LIMAN: No, I saw that during this weekend for the first time, so I am quite happy to take that in as part of the equation but I don't have a statement right now whether it's suitable or not. But absolutely, I am willing to look at what alternatives there already are. It seems like a stupid idea to invent the wheels once more. I would also like to point out already some types†?? DNS Tap, if I understood it correctly, is something that looks at the packet stream and kind of filters out certain properties in the packets.

AUDIENCE SPEAKER: Yes and filter out, it actually adds more information because if you do it in a name server you have more information about zone, how they were structured which you don't have on P CAP. Sorry to interrupt.

LARS?JOHAN LIMAN: That is the direction I was going. There is more information than you can find in the packet itself about internal machinery in the name server and I want that to be ?? I want it to be possible to put that into the various data sets and if DNS Tap has that, that makes it a better starting point. Absolutely.

JIM REID: Any other comments from the panel? No. OK.

AUDIENCE SPEAKER: Mike. This is a sort of a comment for Lars. I am Mark McFadden, Interconnect. And the fact when we think from about statistics for the DNS one of the things there is a lot we don't know that we want to know and I think the name collision thing in the last 12 months has been an example where there are statistics that before we wouldn't have thought of collecting but now, we know we want those statistics and when I thing about what RSAC is doing, what I hope and what I want to pour is flexibility so we don't sort of draw a line in the sand and say for now this is what we are going to collect and find out because of another problem that weified or another issue had a comes up from some place else in the Internet, right, that we need a new set of statistics so this is more a comment rather than a question. What I am encouraging here and supporting is flexibility in the scope of that data set.

LARS?JOHAN LIMAN: Understood. And thank you. That said, I see a difference between the form†?? for shaving information, minimum requirements for collection and route server operators have a special stand here because they have, so to speak, a special obligation towards the general public, but you also have to be careful about how much detail and how much volume you request from the route server operators. That said, I do believe that several groups of operators already collect a lot of data and store it. So it can be retrieved and reused, we saw with the name special collision thing that the data, the bulk data from the Dana life collection that happens once a year was very useful. And I guess we could do more of that, but we can't do every day all over the year, that would be simply too much.

JIM REID: Thank you. If therapeutice are no more comments, I think this probably a good point to sort of close things down, unless any of the panel have anything to add at this stage. Thanks to for taking part in the panel.

And with that, I think we are pretty much done. There are no other items of business and we are only five minutes late. We usually overrun by more than that. I would like to say thanks to everybody, the speakers, some of whom came forward at very short notice to put the agenda together for this section. I would like to say thanks to the audiovisual people and the getting the webcasting organised, the NCC staff and the Jabber rooms and the nice lady doing the stenography. Thank you all for coming and hope to see you in London.