[af-ix-discuss] Renseignements, Interconnexion de deux POP

[Benny.MBOKO at arpce.cg (Benny.MBOKO at arpce.cg)]

Hi Franck,
Thank you for all these explanations.

a CCTLD at the IXP level. 

Question: If the entire CCTLD infrastructure is to be at the IXP level, this infrastructure must be like any other member of the IXP who peering with other. Is it this?

If it is a copy of a CCTLD that needs to be at the IXP level, that copy should just be configured with one of the peering IP addresses. Is it this?

Best Regards
Benny

-----Message d'origine-----
De?: af-ix-discuss [mailto:af-ix-discuss-bounces at af-ix.net] De la part de Frank Habicht
Envoy??: mercredi 10 janvier 2018 20:16
??: af-ix-discuss at af-ix.net
Objet?: Re: [af-ix-discuss] Renseignements, Interconnexion de deux POP

Hi Benny,

What the difference between "interconnection of two different IXPs" and "interconnection of two POPs of the same IXP" depends very much on the definitions, so we probably have to clarify these first.

And the effects of the differences - feasibility etc - then also depend on local circumstances.

Definitions:
First not that some things that call themselves "Internet Exchange" are not really what we here mean by "IXP". One example is the old incumbent operator in South Africa ("SAIX") and another is "EMIX" in the UAE, and there are and were several others.
When you get at one place service to reach the "whole internet" - that's an internet transit service and that's what the above mentioned do. A (possibly big) ISP.

An IXP will allow many different ASNs to connect directly with each other, and does not control the policies or the commercial aspects between them. Typically, an IXP will also not charge depending on the transferred traffic or volume, and also not on a bandwidth subscribed to, other than the physical limitations of the port on the IXP ethernet switch fabric (eg 100M/1G/10G/... or 2x1G/2x10G/..).

If one organisation extends the IXP switch fabric to new location(s), that is in the vast majority of cases within the same city. From one data centre to another data centre. This can most often be because there are some more peers at the second data centre, who want to peer with your existing peers at location 1. And since it would be more expensive for the 10 new peers to each get a fiber to your location 1, it's better the IXP gets 2 (for redundancy) fibers to a seconds switch in location 2.
And then the new switch 2 would also be operated by the IXP operator.
Because the 10 peers at location 2 are competitors to each other and trust the IXP operator more.

So this makes it still one IXP, with multiple "POPs" or "locations"  - almost always in the same city.

If the same IXP would start a new POP in another city, and connect it to the same switch fabric, and transport packets between the cities for the peers, that would make it more like a network operator, they would have to have more support staff, and would be offering services more similar to normal ISPs.

This is a dangerous thing. IXPs depend on the network operators (ISPs, present and prospective peers at the IXP) to view the IXP as neutral and beneficial for the general internet development, and connectivity -- but NOT to compete in the same business and possibly take customers away from the ISPs!

So the distance between those POPs matters.

For two different IXPs to connect to each other...
a) if we assume they are run by the same organisation.
that organisation will in my opinion effectively become a network operator, need support staff, will need to charge differently for traffic staying in city 1 vs traffic crossing the distance to city 2.
It will have to obtain infrastructure or capacity between the cities.
And then what if it will not be fully used? You employ Sales people? You risk paying for a link that is not getting used?

I think it is fair to say that if it will be sustainable, then it's like an ISP business.

Unfortunately in many places in Africa, a (high capacity) link between cities is often more expensive than a Internet transit link of the same capacity at any of the 2 locations. And that gives advantages to content hosted elsewhere, but .... let's not go too fast astray.

Some people argue that with current relatively low traffic levels between locations 1 and 2 we have to build the link, and then traffic will increase, for instance because more content will be hosted locally.
This is the eternal chicken-and-egg problem between local content and local interconnection. And now we are stretching the word "local" very much already - geographically.

Some people think a temporary trial can be useful to find out more; some think someone else can support this as a one-time cost; Would you not better avoid being dependent on that?

So connecting 2 IXPs of the same organisation is like extending one IXP switch fabric to another city. often bad.

b) connecting 2 IXPs of different organisations:
same problems, plus: the intentions and incentives of the 2 organisations might not always stay aligned as much as they are (or
seem) right now.

Is one of them more dominant than the other?
Does the bigger want for the other to pay all the cost of the connection? After all, the smaller gets to enjoy more (all the peers at the bigger IXP).


If there is a border between the 2 locations, it all becomes much more difficult again. Because chances are the 2 regulators do not intend to "enable" (support) connectivity, but to rather restrict and control it :-(


If the border is in a river [ :-) ] then you get more challenges of physical safety and installation of the fiber, but maybe fewer problems with securing right-of-way. If you're not thinking of fiber, than i wish you that the capacity of your physical link will not be enough, and you'll have to upgrade to fiber soon.


If the 2 IXPs try to put the own outdoor plant - fiber or microwave - they will need to get the expertise to do that.
And the expertise is found with staff of telecommunications and ISP companies. Maybe these (at least 2 or 3 of them) should make links across the river, ehm... border, ehm... to the other city ?

And then the ISPs that invested in this can use this as a benefit (and sales pitch) to their customers? and they can also re-sell this to other ISPs. Maybe even at the 2nd IXP location ;-)

If we could get these ideas to the ISPs, if should mean more long-term neutrality and trust in the IXPs.

Are any network operators present at both IXPs?
Are they using an efficient, relatively direct route for their own traffic between these locations?


Thanks,
Frank



On 1/10/2018 7:05 PM, Benny.MBOKO at arpce.cg wrote:
> Hello Nishal,
> First of all, thank you for all the time you have devoted to bring me all this light.
> Could you tell me about the interconnection of two different IXPs and the interconnection of two POPs of the same IXP knowing in the first case that there is only the interconnection link that is shared, whereas in the case of POPs of the same IXP, there is a whole network to build?
> 
> 
> Best regards
> 
> Benny MBOKO
> ARPCE- Congo
> 
> -----Message d'origine-----
> De?: Nishal Goburdhan [mailto:nishal at inx.net.za] Envoy??: mercredi 10 
> janvier 2018 15:40 ??: Benny MBOKO <Benny.MBOKO at arpce.cg> Cc?: 
> af-ix-discuss at af-ix.net Objet?: Re: Renseignements, Interconnexion de 
> deux POP
> 
> On 4 Jan 2018, at 15:07, Benny.MBOKO at arpce.cg wrote:
> 
>> Bonjour ? tous et tous mes v?ux les meilleurs,
>>
>> Je viens vers vous pour un peu plus de lumi?re sur l?interconnexion 
>> de
>> 2 POP.
>> Comment peut-on interconnecter deux Points de Pr?sence d?un IXP 
>> sachant que chaque POP a ses propres ressources (ASN et IP) Les 
>> ressources de management de chaque POP interviennent-elles dans cette 
>> interconnexion ?
>> Quelles sont les ?tapes ? franchir et quelles sont les pr?requis ?
>> Merci pour votre coup de pouce.
>>
>> Cordialement
>> Benny Sh?rif MBOKO OTOKA
>> Administrateur r?seau et syst?me
>> Au Service du tr?s haut d?bit
>> [cid:image001.png at 01D04223.BE54C720]
>> Immeuble ARPCE ,91 bis Avenue de l?amiti? 
>> Centre-ville-Brazzaville-Congo BP : 2490 T?l mobile : (+242) 06 653 
>> 29
>> 38/044525539 T?l bureau : (+242) 05 510 72 72 Site internet : 
>> www.arpce.cg<http://www.arpce.cg/>
>> Email : Benny.MBOKO at arpce.cg<mailto:Benny.MBOKO at arpce.cg>
> 
> 
> hi benny,
> you had sent this message to the mailing list owner in error.  i?m 
> redirecting this to the mailing list itself, (and i took the liberty 
> of replying as well :-)
> 
> happy 2018 to you too!
> 
> it is not a good idea to interconnect internet exchange points.  it is better to allow internet exchange points to run and develop independently.  over time some internet exchange points may fail.  
> that?s not necessarily a bad thing, and simply shows how the market, or policy, changes in a region.  in cases where you hear of IXPs interconnecting, it is generally the same IXP that has multiple PoPs within a city.  this usually happens when there are multiple colocation facilities in the city, and when peering participants (ie. the networks that want to peer) want to be able to connect to the same IX fabric, but from different facilities.
> 
> but, even this is not easy to accomplish, since the IXP administrator would have to take the effort of building infrastructure to connect across the city.  this infrastructure will not be free, and now places the burdens of capital expenditure (ie. equipment) and operational costs (ie. payments to keep the fibre running) on the shoulders of the IXP admin.  generally, IXPs are non-profits that work to improve interconnectivity (and not ISPs that are working to make money) so, increasing the cost of the IXP?s operations is something that you always want to avoid!
> artificially increasing the cost of running the IXP, means that you would need to find creative ways of funding this, and that would quickly distract you, the IXP administrator, from the basic operations of the IXP - which is to run a stable fabric!  so, even if you are thinking of building, or extending your IX to multiple sites, i?d strongly suggest doing this, *if, and only if*, the cost of the fibre between the different locations is very cheap, or free!
> note the first very big constraint here - the cost.
> 
> i started with cost, since, it is important to realise what an IXP is meant to do.  a well-managed IXP will reduce the average-per-bit-delivery-cost (ABPDC) of the networks that connect to it.  ie.  peering makes the network cheaper to run.  if this is not true, networks will stop peering, and continue/start purchasing IP transit instead.  so, if you start to introduce costs into the IXPs operations, the IXP will have to find ways to fund this, and, that goes against the what is probably the first rule of the IXP - which is to make interconnection cheaper!
> of course, the costs go beyond just the recurring costs of the connectivity between the locations;  as the IX operators, if you did decide to build a distributed exchange, you?d need to have a 24x7 NOC to manage this.  and you can?t have just a single person, so that means staff costs, HR costs, etc.  all of which needs to be funded, and all of which makes it even more difficult to get networks to the IX to start.
> 
> there are other reasons as well;  if you do go ahead, and make this happen, then your participants - networks that would otherwise be peering - may complain that you are competing with them for transmission, and may withdraw from your IX.  that is *absolutely* to be avoided, since a network removing itself, is never a good thing for the peering environment.  and, in the long term, this is a direct disincentive to invest in telecoms infrastructure by the private sector;
>   infrastructure, which otherwise, would lead to more competition, lower pricing, and a more stable overall network.
> 
> i?ve mentioned three reasons, and explained just one in some detail.  
> but i?m hoping you start to see that this isn?t a good idea!
> 
> there only a few niche caches where an IXP interconnects directly with 
> another;  these *always* come with limitations;  rules like you are 
> not allowed to use more than X %bandwidth, etc.  in the long run, this 
> situation does not scale, and is generally avoided.  of the more than
> 500 IXPs globally today, there are fewer than ten cases where this happens, and i would strongly encourage you to not follow this model.  
> interconnecting IXPs may sound like a grand idea, and it serves a short-term political goal, but at a practical, and long-term economic level, it?s quite a disastrous thing to do to your economy.
> 
> there are many cities in the world that have multiple IXPs, that do not interconnect with each other.  multiple IXPs in a city work best, when they are considered separate (ie. different infrastructure) and generally only work well, when there?s already a rich mesh of interconnectivity.  in other words, get an IXP up and running;  try to get as many participants connected and peered, and when these participants start to feel like there is a need for resiliency, then it?s a good idea to consider getting another IXP up within the same 
> city.   it?s a *terrible* idea to start two competing IXPs from day1, 
> as this will likely fragment the peering market, and that?s something that you want to avoid at all costs.
> 
> i?ll pause here to give other list folk a chance to express their opinions, but i?m happy to explain this in greater detail if 
> necessary.   ;-)
> 
> ?n.
> 
> ______________________________________________________________________
> This email has been scanned by the IT101 Email Security System.
> For more information please visit http://www.it101.be 
> ______________________________________________________________________
> 
> 
> ______________________________________________________________________
> This email has been scanned by the IT101 Email Security System.
> For more information please visit http://www.it101.be 
> ______________________________________________________________________
> 
> _______________________________________________
> af-ix-discuss mailing list
> af-ix-discuss at af-ix.net
> http://af-ix.net/mailman/listinfo/af-ix-discuss_af-ix.net
> 

_______________________________________________
af-ix-discuss mailing list
af-ix-discuss at af-ix.net
http://af-ix.net/mailman/listinfo/af-ix-discuss_af-ix.net

______________________________________________________________________
This email has been scanned by the IT101 Email Security System.
For more information please visit http://www.it101.be ______________________________________________________________________


______________________________________________________________________
This email has been scanned by the IT101 Email Security System.
For more information please visit http://www.it101.be
______________________________________________________________________