Deal with Network Issues at Layer 3 Using Junos OS

By Walter J. Goralski, Cathy Gadecki, Michael Bushong

Layer 3 issues concern packets, of course. Generally, the higher up in the protocol stack an issue lies, the more things that can be wrong with it. At the routing layer (one early name for Layer 3), routes to a given destination can be absent, can loop, or can send packets into a black hole.

Inside the router network are tools to examine the functioning of the protocols themselves, such as the show route protocol ospf or show bgp summary operational mode commands. You can use these commands to gain insight into the operation of the OSPF and BGP routing protocols, respectively.

However, as you’ve already seen, issues at Layer 3 are often caused by Layer 2 happenings. Even so, you can rely on the SNMP polling and traps and Ethernet OAM to capture problems at the network router link level. Now, take a look at a problem that is actually an issue at the packet layer. Here, you use a typical end user tool — traceroute — to isolate the router causing the problem.

traceroute to find an outage.”/>

Using traceroute to find an outage.

Traceroute sends a packet hop-by-hop from one router to another until the destination host is reached. If a packet arrives at a router that has no route to the destination, a destination unreachable ICMP message is sent back to the originator.

When using traceroute, remember that the problem usually lies not at the last hop to respond to the traceroute, but beyond the last responding device.

When all is well, the destination is normally five hops away from the source host. A change in the responder’s network address is an indication that the packet has moved from one major portion of the network to another (from customer network to service provider network, for example).

In the following, notice how the routers normally respond to a traceroute on the way to Dest-Host:

traceroute to (, 30 hops max, 40 byte packets
1 ( 2.617 ms 1.690 ms 2.851 ms (Cust-Router1)
2 ( 3.386 ms 3.370 ms 5.570 ms (Cust-Router2)
3 ( 13.513 ms 3.905 ms 5.060 ms (Prov-Rtr1)
4 ( 3.778 ms 5.237 ms 5.413 ms (Prov-Rtr2)
5 ( 10.867 ms 12.568 ms 5.991 ms (Dest-Host)

Now, watch what happens if the link — the only link, by the way — between the customer router (Cust-Router2) and the service provider router (Prov-Rtr1) fails:

traceroute to (, 30 hops max, 40 byte packets
1 ( 1.983 ms 2.440 ms 2.414 ms (Cust-Router1)
2 ( 2.883 ms !H 4.136 ms !H 2.114 ms !H

The !H indicates that you are getting ICMP host unreachable messages from the second router. It may look like this Cust-Router2 device is the problem, but notice that the packets have made their way to Cust-Router2 and back without any trouble at all.

No, the issue is beyond this last hop, on the link between the customer and the service provider. There is no useful route to the destination on Cust-Router2, so the !H is issued.