We created a bunch of new LUNs, planning to increase an existing Datastore’s capacity.
VMware side, the operation should be a matter of simply firing up vSphere Client, choosing a host, Configuration tab, viewing the Datastore’s properties, then clicking on the Increase button. Except that no Extent device seems to be found. That’s weird because we already did (multiple times) a rescan of each Storage Adapter/HBA. Moreover, selecting “Add Storage” as if we were to create a new Datastore, indeed shows the expected volumes.
The solution turned out to be this one:

• Connect vSphere Client directly to the host (thus logging in as root), and not to the vCenter Server.

• Headquarter (HQ) connected (MPLS VPN) to some branch sites.
• In some of the branches, a Check Point UTM-1 Edge X (SofaWare) sits between the wireless and wired networks, enforcing security policies between them.
• The two networks are bridged together (Layer 2) by the firewall.
• The wireless LAN is used by some kind of next gen Barcode Scanner: an embedded device with Windows CE .NET 4.2, also able to act as a Terminal Services client.

Customer wants to install some software on the scanners, downloading it from a shared folder residing on one of HQ servers. I add the necessary (and temporary) rules on the firewalls, but the folder still cannot be reached. Windows CE complains that “The network path was not found” but the rules look good.

Luckily, the Edge firewalls provide a packet sniffer, allowing us to further investigate the issue. Just connect to the web based interface of UTM-1/SofaWare, go to Setup → Tools → Sniffer, choose a filter string (using the familiar libpcap/tcpdump syntax), select the interface (“bridge”, in my case), and you’re set. Captured packets can then be downloaded to your PC and opened up in Wireshark.

We came up with a bunch of peculiar NetBIOS Name query requests/answers:

Some hostnames, for clarity:

The Barcode Scanner (Client) asks one of the DNS/Domain Controllers in HQ if it is called HQSERVER. But HQSERVER is the server we’re trying to connect to from the Scanner (by means of \\HQSERVER\sharename)! Why in the world the device should directly ask HQDC1 if it is called HQSERVER? Using an unicast NetBIOS query, too? Obviously HQDC1 answers “no, it’s not me” (Requested name does not exist)… The Scanner then broadcasts the same query to its local network segment, but since HQSERVER sits in Headquarter, it gets no answer and generates the error “The network path was not found”.
Turns out that \\192.168.1.20\sharename causes the same dialoque, with a NetBIOS name query that seemingly asks for a server named “192.168.1.20″. It’s as if in Windows CE, UNC paths could only use names, not IP addresses.

Well, the Customer didn’t have enough time for me to properly solve/understand the issue but we worked around it by:

• Assigning a static IP to the Windows CE device.
• In the TCP/IP settings of Windows CE, use 192.168.1.20 (HQSERVER – where the shared folder is hosted) as DNS and WINS server.
• Copy the needed files from the network share and revert back to DHCP.

Step two makes the Client send NetBIOS name queries to HQSERVER instead of HQDC1. This allows shared folder access to work.

My firewall (the above example comes out from another one) was negotiating 100Mbps speed, Half Duplex. Nothing wrong with that, I tried to fix these parameters on the FortiGate but the Ethernet link would not come up. So, auto-negotiation was mandatory and I had no way to change that on the router.
At some point, when Internet connectivity was stuck, it seemed to me that unplugging and plugging back in the cable between firewall/router, would allow for a faster recovery. Definitely, something was wrong at L2.
The solution was to insert a 15€ DLink switch between firewall and router. No problems since then, it really looks like FortiGate and Cisco NICs don’t play well together, at least in that conditions. The Customer will call the ISP in order to tweak the settings Cisco side and see if they can get rid of the switch.
The proper way to diagnose the problem would’ve been to ping the router from the outside during a connectivity stop. Since the issue was “local”, the router should answer while no traffic should pass from the firewall to the router.