In data centers, customers are interconnected with carriers and other customers using dedicated physical connections called cross-connects. These can be just about any type of medium, however most cross-connects run inside a single building are copper or fiber Ethernet. In the simplest of cases, a cross-connect can be a single CAT 5e or 6 cable running from a patch panel in one cage to a patch panel in another cage. Very little can go wrong, or so one might think.

I ordered two such Gigabit Ethernet cross-connects recently. Two lines of CAT 6 cable from a cage leased by company to a customer's cabinet down the hall. Simple. One of the cross-connects came up fine, as usual. The other did not.

This was odd, because the data center tech who installs the cross-connect is responsible for certifying its operation before making it available to the customer. I check it out with my Fluke and see that pairs 3/6 and 4/5 are crossed. No big deal, probably just needs an end re-terminated. I disconnect the patch cables from the panels at either end of the cross-connect so a tech can re-terminate it, open a trouble ticket with the data center, and go on about my day.

The next day, I get an email confirmation that ticket with the data center has been closed. Awesome. I go to plug the cables back in thinking the issue has been resolved. Same problem as before; a short on 3/6 and 4/5. Annoyed, I call up the data center help desk and ask what's up.

It seems that the technician dispatched to handle my ticket immediately concluded that my cross-connect was not working because "the patch cables were unplugged at both ends."

The data center has a very responsible policy of never disconnecting a plugged-in cable without first contacting a customer it belongs to for verification, hence me leaving the cross-connect patches disconnected. After conveying this logic to the help desk representative, I scheduled a second visit, this time requesting a tech to meet me in the cage at a specific time should any hand-holding be necessary.

The appointment comes around the next week and I head out to the data center. I wait in the cage for about ten minutes, and no one else shows up. I call up the help desk, again, and ask them to send someone out. The tech shows up maybe ten minutes later. In the cage behind mine. I suppose they do all look alike, though only mine has the cage number I gave him ten minutes ago.

The tech makes it into the cage and starts working. Breaks out his Fluke cable tester (one of the fancy models) and runs a TDR test on the cable. The Fluke randomly reboots halfway through the test. He tries again, it reboots again. He leaves to grab another tester. This one completes the test and the cross-connect fails verification.

He dismounts the jack from the panel and finds the original installer neglected to trim the wire ends after punching down the CAT 6 cable. Sloppy work, but an easy fix, and likely the cause of the short. The tech trims the stray wires from both ends of the cross-connect. Hooray, progress! Or so I thought.

The tech goes to mount the jack back in the panel, which is at a difficult angle, and breaks the jack. Oops. He reterminates the cable into a new jack and successfully mates it back to the patch panel. Now we're getting somewhere. I thank him and plug in the patch cable on that end. I head over to the cabinet where the other end of the cross-connect terminates, intent on plugging in the patch at that end as well and calling it a day.

Except I can't open the cabinet door. At some point in the last few days, the electronic locking mechanism on the cabinet door died. The cabinets on either side of this one open fine, as does the door on the opposite side of this cabinet. But not this door. Not the only door I need to open for ten seconds so I can plug in a cable and put this all behind me.

I call up the help desk again. If the same person answered the phone every time, we'd be on a first-name basis by now. I explain my predicament. She says it's the first time she's ever encountered this issue in her employment there. Lucky me. She puts in a high-priority smart hands ticket and I chill for a bit in the pseudo-office space they have for customers.

A different data center tech pops in to verify the problem and cabinet number. I hesitantly inquire as to how they even set about getting open a door with a failed lock, and my fears are confirmed: a crow bar. This explains the bent cabinet doors I see leaning up against cage walls here and there throughout the data center.

I received confirmation this afternoon that my cabinet door issue has been... resolved. But to be completely honest, I'm no longer sure I want to plug in this cross-connect. The amount of hassle this one particular cross-connect has conjured cannot be coincidental. It's as if some divine power is forbidding this cross-connect from ever carrying data. But dammit, I'm determined to win this battle. We'll see how it goes tomorrow.