NSX Packet Traversal: Understanding Ingress, Egress, and the Journey Between a Virtual Machine and the Physical Network

TL;DR

VMware NSX packet traversal becomes easier to understand when you follow traffic across its actual forwarding and security boundaries. An external request may enter through a physical router, reach an NSX Edge node and Tier-0 gateway, cross a Geneve overlay tunnel, encounter distributed security enforcement, and arrive at an application virtual machine. The response must follow a valid return path, potentially involving different routing, Network Address Translation (NAT), and stateful security considerations.

The accompanying 18-second animation illustrates ingress and egress across a VMware Cloud Foundation (VCF) networking environment. It represents a centralized NSX Virtual Private Cloud (VPC) connectivity design. Other supported architectures, including Distributed Transit Gateway designs, can use substantially different north-south forwarding paths without an NSX Edge cluster.

For architects and engineers, the important question is not simply whether a packet is allowed through a firewall. It is where the packet is forwarded, where its headers may change, where policy is enforced, which physical paths carry it, and what evidence proves successful delivery.

Key takeaway: NSX connectivity should be validated as an end-to-end forwarding path across the workload, distributed datapath, gateway services, overlay transport, and physical network, in both directions.

Introduction

An application team reports that its production virtual machine is running, its network adapter is connected, and the firewall rules appear correct. Yet external clients cannot establish a connection.

The NSX dashboard looks healthy. The physical network team sees active switch interfaces. The security team sees an allow rule. Everyone has evidence that their individual component is functioning, but nobody has established where the packet actually stops.

This is a common failure in virtualized networking troubleshooting. The environment is evaluated component by component instead of as a connected forwarding system.

VMware NSX makes networking more flexible by separating logical network services from their physical transport. However, that separation introduces multiple locations where routing, encapsulation, network address translation, security policy, and physical forwarding can influence the same connection.

The video accompanying this article follows that relationship from an application virtual machine through NSX virtual networking, an Edge cluster, and the physical network.

The goal here is to turn the animation into a practical packet-traversal model that architects and operators can use for design reviews, troubleshooting, and production validation.

Scope: The primary example is a VMware Cloud Foundation 9.1 VPC using a Centralized Transit Gateway (CTGW) external connection. The article also distinguishes Distributed Transit Gateway (DTGW) forwarding. Exact forwarding and service behavior depend on the installed NSX release, network topology, routing configuration, security policies, and optional services.

What the NSX Packet Traversal Video Actually Shows

The 18-second animation moves between two perspectives:

• Ingress: Enterprise Network / Internet to Application VM.
• Egress: Application VM to Enterprise Network / Internet.

The left side represents the virtual workload environment. The middle contains the NSX Edge and centralized gateway boundary. The right side represents physical forwarding through the top-of-rack or leaf switches, core routing, and external networks.

The most useful part of the animation is the relationship between those regions.

The video contains several distinct networking concepts:

There is one essential interpretation rule.

The illustration is a logical architecture visualization, not a literal series of independent physical appliances.

The distributed switch, Distributed Firewall, and distributed routing functions are implemented in the host networking datapath. The video separates them visually to explain their responsibilities.

Likewise, the displayed sequence is an educational model, not a packet capture proving that an observed production flow visited every represented object in that exact order.

That distinction becomes important when troubleshooting. An engineer should use the animation to identify possible forwarding and policy boundaries, then confirm the actual datapath in the relevant environment.

Following an NSX Packet Through the Centralized Architecture

The following diagram translates the video into two directional paths. It represents a typical north-south flow involving a centralized transit connection.

The diagram simplifies several implementation details. The actual NSX datapath, lookup sequence, and service placement depend on the routing and security configuration. Some logical operations occur on the host, while centralized services execute on the Edge. Geneve encapsulation applies to an overlay tunnel between relevant TEPs, not to every packet at every stage.

Ingress: From the External Network to the Virtual Machine

Imagine an external client connecting to an application hosted inside an NSX VPC.

The first forwarding decisions happen before the packet reaches the workload.

Physical network arrival

The external network routes the packet toward the destination prefix or published application address. Depending on the enterprise design, the packet passes through core routing and a leaf or top-of-rack switch.

For a centralized VPC external connection, that physical path must eventually reach the appropriate NSX Tier-0 uplink.

The physical networking team must establish the required routing adjacency or static routing, address reachability, and uplink availability.

NSX Edge and Tier-0 processing

The packet enters the NSX environment at the centralized external boundary.

The Tier-0 service-router path and associated transit networking provide connectivity toward the correct virtual destination. Applicable gateway security services, NAT, and other configured services may influence the packet before it reaches the workload overlay.

For example, an application published through an externally reachable address may require destination NAT (DNAT) to translate that address into a private workload address.

Not every ingress path requires NAT. A workload subnet using appropriately routed addresses may be reachable without that translation.

Transit and VPC routing

The forwarding logic determines which VPC and subnet should receive the traffic.

The VPC Gateway supplies distributed routing for VPC subnets, while the Transit Gateway provides connectivity between VPC gateways and approved external connections.

A centralized design may require the packet to cross from the Edge to the ESX host through an NSX overlay tunnel.

The physical network transports the encapsulated traffic between the relevant endpoints. At the destination transport node, the overlay encapsulation is removed and the workload packet continues through the virtual datapath.

Distributed security enforcement

Applicable NSX Distributed Firewall rules are evaluated at the protected workload’s virtual interface.

This is an important security distinction. Reaching the correct host or logical segment does not establish permission to reach the application.

An ingress flow might successfully pass external routing and gateway policy, only to be blocked by a Distributed Firewall rule near the destination VM.

Workload delivery

If the packet is permitted and correctly forwarded, it reaches the VM’s vNIC and the guest networking stack.

The guest operating system, local firewall, application listener, and application-level security controls still determine whether the connection succeeds.

A packet delivered to a VM does not automatically mean the application accepted it.

Egress: From the Virtual Machine to the External Network

Egress starts at the workload.

An application sends a packet to an address outside its local network. The operating system identifies its next hop and sends the packet through the virtual network adapter.

The host datapath processes the traffic through the applicable logical switching, routing, and distributed security functions.

For an external destination using a centralized transit connection, forwarding sends the traffic toward the relevant centralized Edge service-router path.

If the selected route requires traversing the overlay to the Edge, the host TEP encapsulates the traffic in Geneve and sends it toward the Edge TEP across the physical IP network.

The Edge processes applicable services. If outbound source NAT (SNAT) is required, the packet’s source address is translated to an approved external address before the packet leaves the virtual networking environment.

The packet then exits through the NSX external uplink and is forwarded by the physical network.

Egress is not guaranteed simply because ingress works, and ingress is not guaranteed because egress works.

Routing decisions, firewall policy, address translation, and application state can differ between directions. A working outbound connection does not prove that the workload has a valid inbound publication path.

The Overlay and Underlay Are Two Different Forwarding Problems

One of the strongest visual elements in the video is the separation between the blue NSX virtual overlay and the green physical underlay.

They work together, but they solve different problems.

What the Overlay Provides

The NSX overlay allows workload networking to be defined independently of the physical network’s ordinary VLAN and IP addressing structure.

A virtual machine connects to a logical segment. NSX maps the workload’s traffic into the applicable logical forwarding environment.

When traffic needs to cross an overlay tunnel between transport nodes, Geneve encapsulation carries the workload packet inside an outer transport packet.

The outer packet uses transport-network addresses to reach the destination TEP.

This gives the physical network a relatively straightforward responsibility: provide reliable IP connectivity between the tunnel endpoints.

What the Underlay Provides

The physical underlay carries packets between ESX hosts, Edge transport nodes, and the relevant physical switching and routing infrastructure.

For overlay transport, the underlay must support the required TEP reachability and encapsulated packet sizes. Uplink design, physical routing, MTU configuration, and link redundancy matter.

A correctly configured VPC does not compensate for an underlay that drops encapsulated packets.

For example, a small test packet may succeed while larger application traffic experiences loss if packet sizing and MTU support are inconsistent along the relevant path.

That does not prove an MTU issue is present. It identifies a condition worth validating when failures correlate with packet size or workload behavior.

Understanding the Encapsulation Boundary

A useful troubleshooting distinction is that packet headers visible at different locations may represent different layers.

A physical switch capture of an encapsulated overlay packet may primarily expose the transport endpoints unless the capture is decoded.

Likewise, seeing traffic on an Edge uplink does not establish that the destination application received it.

The engineer must know which version of the packet is visible at each observation point.

The Architecture Changes When the Transit Gateway Is Distributed

The video deliberately illustrates an NSX Edge cluster. That makes it a useful representation of centralized external connectivity.

However, VMware Cloud Foundation 9.1 also supports distributed external connectivity patterns in which an Edge cluster is not required for the north-south forwarding path.

That architectural choice changes where the packet travels and where operators should investigate failures.

VMware introduced additional flexibility in the VCF 9.1 generation, including more external connection options and service capabilities through Virtual Network Appliance (VNA) clusters for applicable distributed designs.

This does not mean the two architectures have identical features, service placement, or failure characteristics.

For engineers, the immediate implication is straightforward: do not start an NSX north-south troubleshooting exercise by assuming every packet must traverse an Edge node.

First establish which external connectivity model is configured.

In a distributed design, looking for a nonexistent Edge forwarding hop wastes time. In a centralized design, ignoring Edge services and routing can leave the actual problem undiagnosed.

Firewall Enforcement and NAT Change the Meaning of Packet Traversal

The animation includes both distributed firewalling and Edge services because routing alone is insufficient to explain successful application connectivity.

A packet can follow a valid route and still be denied.

It can also be allowed by one policy domain but denied by another.

Distributed Firewall: Enforcement Close to the Workload

The NSX Distributed Firewall is enforced in the distributed workload datapath.

This placement enables security policy to be associated with workloads rather than relying exclusively on perimeter addresses and centralized inspection devices.

A protected application VM might have rules governing connections from a web tier, management systems, application dependencies, and external clients.

The applicable enforcement depends on the rule scope, groups, policy configuration, and traffic direction.

The DFW is therefore a distinct control from the physical network’s routing and any centralized gateway firewall.

Gateway Firewall: Enforcement at the Relevant Routing Boundary

Gateway Firewall policy can provide additional controls on traffic traversing the associated gateway or service-router path.

In centralized designs, this introduces another security enforcement location that must be accounted for during incident investigation.

An allow decision at the gateway does not override an applicable distributed firewall deny.

Conversely, allowing traffic at the workload boundary does not make it immune from an upstream gateway policy.

Security teams should be able to determine which policies govern a flow, where those policies are enforced, and which rule produced an observed decision.

NAT: When Packet Identity Changes

NAT is particularly easy to overlook in an architecture animation.

Consider an illustrative inbound HTTPS connection:

External client: 198.51.100.25
Published IP:    203.0.113.40
Application VM:  10.44.8.20

Before DNAT:
198.51.100.25:51234 -> 203.0.113.40:443

After DNAT:
198.51.100.25:51234 -> 10.44.8.20:443

These are documentation-only example addresses, not observed values from the supplied video or a deployed environment.

The example illustrates why a physical firewall, gateway, and workload firewall may not all observe the same destination address.

During the return flow, the translation state must be applied consistently so the external client sees the expected published endpoint.

Depending on the actual design, address translation may take place at different supported service locations.

A troubleshooting record must therefore preserve the original and translated flow information, the relevant rule or mapping, and the observation point.

Proving the NSX Packet Path with Operational Evidence

A useful way to operationalize the video is to convert the displayed architecture into a sequence of diagnostic checkpoints.

Rather than collecting every log in the environment, start with the application connection and identify where its last confirmed forwarding observation occurred.

The following map separates five useful checkpoints.

This is an evidence map, not a mandatory list of physical devices. In a distributed transit design, the Edge checkpoint is replaced by the actual host-to-fabric forwarding and optional service path.

Build an Observation Record

For each checkpoint, determine what evidence is available and what it proves.

The objective is to identify the last confirmed good observation and the next unproven boundary.

That narrows the investigation without assigning blame based on which team owns the component.

Use NSX Traceflow to Validate Logical Forwarding

VMware NSX Traceflow is particularly useful here.

Traceflow injects synthetic packets and collects observations from NSX components along the supported datapath. It can help identify forwarding decisions, security drops, routing behavior, and applicable translations.

For a centralized north-south ingress scenario, VMware’s published Traceflow guidance describes testing from a Tier-0 uplink toward a virtual destination.

The engineer should select the relevant source and destination, protocol, and packet attributes, then inspect the returned observations.

Source routing and reverse-path forwarding requirements matter. An incorrect simulated source or missing route can produce a result that reflects the test setup rather than the original application problem.

A Traceflow result indicating that traffic was delivered toward the physical network must also be interpreted carefully.

It does not prove that the packet reached its final external destination.

Physical routers and switches outside NSX do not generally report their forwarding observations into the NSX Traceflow result. A successful virtual-network handoff may still be followed by a physical routing, filtering, or destination-side failure.

Traceflow should therefore be combined with appropriate physical networking evidence and actual workload testing.

Use Packet Captures When You Need to Inspect Real Traffic

Traceflow is not a substitute for every packet capture.

Actual captures become important when the problem involves retransmissions, asymmetric paths, packet-size behavior, transport-state problems, application-layer exchanges, or differences between synthetic test traffic and live sessions.

The capture point must match the question.

A guest capture can establish whether the workload sends a response. A virtual datapath capture may establish whether the response reaches the appropriate NSX forwarding stage. An Edge-side or physical capture can establish whether traffic crosses a specified boundary.

Some capture operations can generate substantial data or expose sensitive payloads. Use an approved change or diagnostic procedure, limit collection to the necessary interfaces and flows, and preserve evidence appropriately.

A Practical Example: Outbound HTTPS Works, but Inbound HTTPS Fails

Consider an application hosted on a private VPC subnet.

The application can successfully establish an outbound connection to a software update service. However, external clients cannot connect to its published HTTPS endpoint.

The application team argues that the network is functioning because outbound internet access works.

That conclusion is too broad.

Outbound connectivity may depend on a default route, an SNAT configuration, and an outbound security policy. Inbound access may depend on an advertised or otherwise reachable external destination, DNAT or another publication mechanism, an applicable ingress security policy, and the application accepting connections.

The two flows share infrastructure, but they do not necessarily share the same access requirements.

Investigate the Inbound Failure

Begin by confirming the expected destination address and TCP port from the external client’s perspective.

Next, verify that the physical network can reach the appropriate centralized external boundary. For the illustrated design, this means checking the relevant Tier-0 path and upstream routing.

Examine the applicable NAT or application-publication configuration if the published address differs from the workload’s actual IP address.

Then validate logical forwarding through the transit and VPC networking layers.

Review the relevant NSX security decisions, including gateway and distributed policy when configured.

Finally, determine whether the guest receives the connection attempt and whether its local firewall and application listener accept it.

The first confirmed drop or missing observation determines where the next investigation should focus.

Validate the Return Path

The next question is whether the application can send its response back through a valid route.

A destination can receive a TCP SYN but fail to return a SYN-ACK because of guest configuration, local policy, routing, stateful security behavior, or a broken network path.

If NAT is involved, confirm that the return flow is translated as expected.

Also consider whether load sharing, failover, or routing policy causes related packets to take paths that cannot preserve the necessary connection state.

A successful outbound connection initiated independently by the VM does not settle any of these return-path questions.

Establish a Defensible Resolution

Suppose the investigation reveals that the external address is reachable and correctly translated, but the corresponding inbound connection is denied at a workload policy boundary.

The appropriate correction is a narrowly scoped change to the applicable policy, subject to security approval and validation.

It is not a broad temporary allow rule across the entire VPC.

After the approved change, the team should repeat the original client-to-application test, verify the response, confirm effective policy behavior, and record the result.

The important outcome is that the team can explain precisely why the connection works, not merely observe that a test eventually succeeded.

Common NSX Packet-Flow Failure Patterns

The video shows a clean, successful conceptual path. Production environments need to account for where that path can break.

These are diagnostic hypotheses, not universal symptom-to-cause mappings. An actual incident still requires observed evidence before a cause is confirmed.

What the Animation Means for NSX Architecture and Day-2 Operations

Packet traversal is an engineering concern during initial design, but it becomes an operating-model concern after deployment.

The architecture needs enough documentation that teams can reconstruct a connection without relying on the engineer who originally configured it.

Define the Network Ownership Boundaries

In a VPC operating model, the provider generally owns external connectivity definitions, physical integration, and the platform network infrastructure.

Tenant administrators consume approved virtual networking constructs and manage the appropriate VPC, subnet, routing, and service configuration within their delegated scope.

Security and application ownership may be separate again.

An organization should document who owns the Distributed Firewall policy, gateway security policy, NAT changes, route advertisements, transport-node health, and physical switching.

Without that distinction, a connectivity incident can involve several teams independently proving that their individual configuration is correct.

Validate Both Directions During Changes

Network changes should be tested against representative inbound and outbound flows.

A gateway change may affect external connectivity. A firewall update may interrupt application dependencies. A subnet or routing change may affect forwarding. An Edge maintenance event may change the available service path. A workload migration may expose a missing host or fabric prerequisite.

The change record should identify which flows might be affected and what observations establish successful operation after the change.

A healthy NSX management cluster is valuable evidence, but it does not prove every application path is healthy.

Treat Encapsulation and Physical Connectivity as Shared Dependencies

Network architecture documentation should explicitly distinguish the transport network used by overlay TEPs from external routing and uplink connectivity.

Although both depend on physical networking, they serve different forwarding purposes.

Teams should understand which paths carry encapsulated Geneve traffic, which paths carry externally routed workload traffic, and where those traffic types share underlying links.

Capacity, MTU, redundancy, and maintenance plans must reflect those dependencies.

Record the Evidence Needed for Recovery

The most useful operational record for a critical application includes its logical segment and VPC, effective external connection, network addressing and NAT expectations, applicable firewall ownership, expected routes, and relevant physical dependencies.

Combine that with known-good connectivity tests and a repeatable escalation procedure.

When the network fails, the operations team can compare the present behavior against a documented baseline rather than reconstructing the architecture during an incident.

Conclusion

The NSX packet traversal video provides a compact mental model for a complex part of modern VMware Cloud Foundation networking.

An application packet begins or ends at the virtual machine, but its successful journey depends on several independently managed functions: the guest network stack, distributed switching and security, VPC routing, transit connectivity, overlay transport, optional gateway services, and the physical network.

The centralized architecture shown in the video is one valid implementation pattern. Distributed connectivity can remove the NSX Edge from the external forwarding path, shifting additional responsibility toward host connectivity and physical fabric integration.

The most important operational lesson is that packet traversal must be understood in both directions. Routing, security, NAT, encapsulation, and actual packet delivery require separate forms of evidence.

A well-designed NSX environment should make those boundaries understandable before an incident occurs.

The practical question for the next architecture review is simple:

When a critical application connection fails, can the team identify the last confirmed forwarding point, the next unproven boundary, and the owner responsible for validating it?

If the answer is yes, the animation has become more than a visual explanation. It has become a useful operating model.

External References

• VPC Centralized Network Connectivity With Guided Edge Deployment (VMware Cloud Foundation Blog)
• VMware Virtual Private Cloud in VMware Cloud Foundation 9.0 (VMware Cloud Foundation Blog)
• Transit Gateway Connectivity Options in VCF 9.1 (VMware Cloud Foundation Blog)
• Simplify Workload Connectivity and Enhance Network Scale and Performance with VCF 9.1 (VMware Cloud Foundation Blog)
• 5 Things You Didn’t Know You Can Do With Traceflow (VMware Cloud Foundation Blog)

The post NSX Packet Traversal: Understanding Ingress, Egress, and the Journey Between a Virtual Machine and the Physical Network appeared first on Digital Thought Disruption.