VPN on FortiGate

Documentation for configuring VPN tunnels on FortiGate units (in English) can be found on the manufacturer’s website at https://docs.fortinet.com/document/fortigate/7.2.3/administration-guide/954635/getting-started

and https://fortinetweb.s3.amazonaws.com/docs.fortinet.com/v2/attachments/a06117ca-5fbf-11ed-96f0-fa163e15d75b/FortiOS-7.2.3-Administration_Guide.pdf

Access data for the FortiGate cluster

VDOM access in the Fortigate cluster can be found in the Atman Cloud panel - https://panel.atman.pl/my-cloud. Select the Cloud Environments tab and then the VPN tab.

pic1

If there is no VPN tab in the panel, the VPN service has not been activated for your environment. To activate it, please contact your account manager.

Access to the VDOM panel

The VDOM panel is accessible only from IP addresses registered with Atman. If the VDOM login page does not open (connection timed out), the most common cause is that you are connecting from an address not on this list.

To add or change IP addresses, please send a request to servicedesk@atman.pl and provide the following:

  • the name of the Atman Cloud service,

  • a list of IP addresses in the format x.x.x.x/32,

  • addresses to be removed from the list.

The request must be submitted by a person authorised to manage the service. Otherwise, Atman will ask the account administrator for confirmation.

VDOM limits

The number of tunnels, firewall policies and address objects in a VDOM is limited. Once the limit is reached, attempting to add another item will result in the message The VDOM property limit has been reached. To increase the limit, please contact your account manager.

Instructions for migrating OpenStack VPN tunnels to a FortiGate cluster

In the new solution, the customer gets one virtual domain (VDOM) on the Fortigat cluster with a separate public IP address. The VDOM cluster is accessed via the https protocol (https://<address_IP_vdom>) and ssh. After logging into the VDOM, the connection address between the FortiGate and the cloud environment (cloud router) and the routing setting on this router must be configured. An overview of the connection architecture of the new solution is shown below.

pic2

Configuring the connection between the FortiGate cluster and the cloud environment

On the OpenStack side, we get a link between the cloud and vdom in the Fortigat cluster. The connector will be visible in the Network -> Networks tab. The network will be added and will have CROSSCONNECT in its name.

pic3

In our connection network, we configure the connection address between the cloud router and the vdom in the FortiGate.

pic4

We then fill in the form below. In our example, we have used the address 10.255.255.0/30 but it can also be any other address.

pic5

pic6

PLEASE NOTE

In the Subnet Details tab, the Enable DHCP option must be unchecked. When DHCP is enabled, OpenStack assigns the first available address from the pool (in the example, 10.255.255.2) to the port handling DHCP and metadata. If the same address is set on the FortiGate interface, the tunnel will be established, but traffic between the cloud and the VPN will not pass through – see Tunnel is established, but traffic is not passing through.

We add the resulting network to the router in OpenStack and set the address on the FortiGate internal interface. To do this, we go to the Network -> Routers tab, then select our router and add the ADD INTERFACE interface.

pic7

In our example, we will set the address 10.255.255.1 on the router side of OpenStack and the address 10.255.255.2 on the FortiGate side.

pic8

In the Static Routes tab of our router, we add the routes to be reachable via vpn on the FortiGate.

pic9

Below is an example of adding a route for the prefix 10.12.0.0/24 to be reached by the FortiGate. The address 10.255.255.2 is the address we have set on the FortiGate.

pic10

Once we have created the configuration on the cloud environment side then we move on to the configuration on the FortiGate side. To complete the connection to the cloud, we need to set the connection address on the Internal interface. In our case 10.255.255.2/30.

pic11

pic12

Below we set the connection address. For this we select Addressing mode Manual and IP/Netmask .

pic13

PLEASE NOTE

On the FortiGate internal interface, set only an address from the CROSSCONNECT network. Do not add any additional addresses (Secondary IP Address) from the cloud network, e.g. 192.168.20.1/24. FortiGate will then treat this network as being directly connected and will not use the route via the router in OpenStack.

Adding routing on the FortiGate side. Here we add networks that are in a cloud environment and we want to reach them via VPN.

pic14

pic15

Creation of VPNs

VPN tunnels can be created, for example, in the VPN -> IPsec Wizard tab.

pic16

Complete the format. Remote IP Address is the address of the tunnel on the other side, pre-shared key is the key which must be the same on both sides of the tunnel.

pic17

In the next format, we fill in the addresses to be sent through the tunnel. In this case 10.255.255.0/30 and 192.168.20.0/24 are to be terminated on the cloud side. On the other side of the tunnel, the address 10.12.0.0/24 is to be located.

pic18

We then click Create and the tunnel configuration is created.

pic19

The wizard creates a tunnel, address objects, a static route to the network on the other side of the tunnel, and two firewall policies (from the cloud to the tunnel and from the tunnel to the cloud).

Recommended tunnel parameters

The parameters below are an example of a safe default configuration. The most important thing is to set the same values on both sides of the tunnel.

Parameter

Phase 1

Phase 2

IKE version

IKEv2

–

Encryption

AES256

AES256

Authentication

SHA256

SHA256

The Diffie–Hellman group

14

14 (PFS enabled)

Key Lifetime

28,800 s

3,600 s

Dead Peer Detection

On Idle

–

Auto-negotiate

–

enabled

Phase 2 selectors (local and remote networks) must be mirrored on both sides: the local network on the FortiGate side is the remote network on the other side, and vice versa, with the same subnet mask.

If several networks are to be tunnelled, and the other end is, for example, a Cisco ASA, Juniper, MikroTik or Azure, we recommend creating a separate Phase 2 selector for each pair of networks rather than a single selector with a group of addresses.

Troubleshooting

Most tunnel-related issues stem from inconsistent configurations on either side of the tunnel, or from missing routing entries or policies. The sections below describe how to check the tunnel status, identify the cause of an error and resolve it.

The following addressing conventions have been used in the examples:

Element

Address

the network on Atman Cloud

192.168.20.0/24 (instance 192.168.20.10)

the CROSSCONNECT network

10.10.0.0/24 – router in OpenStack 10.10.0.1, FortiGate 10.10.0.254

the network on the other side of the tunnel

10.12.0.0/24

tunnel

VPN Office

Checking the tunnel status

IPsec Monitor

The status of the tunnels is displayed on the Dashboard -> Network screen, in the IPsec widget. Clicking on the widget opens a detailed view.

pic20

  • Phase 1 column – status of Phase 1 negotiations (authentication of the parties),

  • Phase 2 Selectors column – status of Phase 2 selectors (networks transmitted via the tunnel),

  • Incoming Data / Outgoing Data – traffic received and sent via the tunnel. If only one of the values is increasing, traffic is only passing through the tunnel in one direction,

  • Bring Up / Bring Down buttons – manual establishment or termination of the tunnel,

  • Show Matching Logs – event logs relating to the selected tunnel.

Below is a tunnel where phase 1 is up but phase 2 is not. The most common cause is mismatched selectors – see Phase 2 does not come up.

pic21

CLI console

CLI commands can be executed without an SSH connection – in the VDOM panel, click the >_ icon in the top right-hand corner. The CLI Console window will open. You can save the command output to a file by clicking the Download button.

pic22

Tunnel status summary:

get vpn ipsec tunnel summary

'VPN-Biuro' 77.79.246.39:4500  selectors(total,up): 1/1  rx(pkt,err): 200/0  tx(pkt,err): 139/0

The value selectors(total,up): 1/1 indicates that all Phase 2 selectors are set. The value 1/0 indicates that Phase 2 is not set.

VDOM routing table:

get router info routing-table all

Tunnel details:

diagnose vpn ike gateway list name <tunnel_name>

diagnose vpn tunnel list name <tunnel_name>

PLEASE NOTE

The output of the commands diagnose vpn ike gateway list and diagnose vpn tunnel list contains the encryption keys for the current session (the lines SK_ei, SK_er, SK_ai, SK_ar, dec: and enc:). Before sending the output to anyone, remove these lines.

VPN event logs

Event logs are available on the Log & Report -> System Events screen, after selecting VPN Events from the list. The reason for the error is displayed in the Reason field once you have selected the row containing the error and clicked Details.

pic23

Messages that most commonly indicate the cause of the problem:

  • IPsec phase 1 error, Reason probable preshared key mismatch – the pre-shared keys on the two sides differ. Set the same key on both sides.

  • IPsec phase 1 error, Reason peer SA proposal not match local policy – the encryption algorithm, authentication algorithm, DH group or IKE version in phase 1 differ. See The tunnel does not come up (phase 1).

  • IPsec phase 2 error, Reason peer SA proposal not match local policy – the algorithms or the PFS setting in phase 2 differ. Set the same phase 2 parameters on both sides.

  • IPsec DPD failed, Status dpd_failure – the other side has stopped responding (device failure, IP address change, UDP 500/4500 blocked). Check the device and the connection on the other side.

Mismatched phase 2 selectors are not visible in the event logs – phase 1 completes successfully, but phase 2 does not come up. The cause is shown only in the negotiation log described below.

Detailed negotiation log

If the event logs do not indicate the cause, you should enable IKE negotiation logging for the other end of the tunnel in the CLI:

diagnose vpn ike log filter clear
diagnose vpn ike log filter dst-addr4 <publiczny_IP_drugiej_strony>
diagnose debug application ike -1
diagnose debug console timestamp enable
diagnose debug enable

Next, establish the tunnel (e.g. using the Bring Up button in the IPsec monitor) and read the messages. Once the diagnostics are complete, disable the log:

diagnose debug disable
diagnose debug application ike 0
diagnose vpn ike log filter clear

Messages in the negotiation log and their meanings:

  • PSK auth failed: probable pre-shared key mismatch – the pre-shared keys differ. The tunnel negotiation was started by the other side.

  • received notify type AUTHENTICATION_FAILED – the other side rejected the authentication, most often because of a different pre-shared key. The tunnel negotiation was started by the FortiGate.

  • no proposal chosen – the phase 1 or phase 2 parameters differ. Compare the incoming proposal and my proposal sections.

  • Negotiate ISAKMP SA Error and no SA proposal chosen – the other side uses a different IKE version (e.g. IKEv1 while IKEv2 is set on the FortiGate).

  • failed to match peer selectors – the phase 2 selectors do not match.

  • repeated sent IKE msg (RETRANSMIT_SA_INIT) with no response – the other side does not respond: wrong Remote Gateway address, UDP 500/4500 traffic blocked on the other side, or the tunnel disabled on the other side.

  • send IKEv2 DPD probe and link fail – the other side stopped responding on an established tunnel (Dead Peer Detection).

An example for different Diffie-Hellman groups in phase 1 – the other party proposes group 15 (MODP3072), whilst group 14 (MODP2048) is set on the FortiGate:

ike 2:...: incoming proposal:
ike 2:...:         type=ENCR, val=AES_CBC (key_len = 256)
ike 2:...:         type=INTEGR, val=AUTH_HMAC_SHA2_256_128
ike 2:...:         type=PRF, val=PRF_HMAC_SHA2_256
ike 2:...:         type=DH_GROUP, val=MODP3072.
ike 2:...: my proposal, gw VPN-Biuro:
ike 2:...:         type=ENCR, val=AES_CBC (key_len = 256)
ike 2:...:         type=INTEGR, val=AUTH_HMAC_SHA2_256_128
ike 2:...:         type=PRF, val=PRF_HMAC_SHA2_256
ike 2:...:         type=DH_GROUP, val=MODP2048.
ike 2:...: no proposal chosen

An example of incompatible selectors – the other side reports the network 10.13.0.0/24, whilst 10.12.0.0/24 has been set as the remote network on the FortiGate:

ike 2:VPN-Biuro:...: TSi_0 0:10.13.0.0-10.13.0.255:0
ike 2:VPN-Biuro:...: TSr_0 0:192.168.20.0-192.168.20.255:0
ike 2:VPN-Biuro:...: comparing selectors
ike 2:VPN-Biuro:...: failed to match peer selectors

The tunnel does not come up (phase 1)

Please check that identical values are set on both sides of the tunnel:

  • Remote Gateway address (the other party’s public IP address; on the other side – the VDOM’s public IP address),

  • IKE version (IKEv1 or IKEv2),

  • pre-shared key,

  • encryption algorithm, authentication algorithm and Diffie-Hellman group,

  • key lifetime (Key Lifetime),

  • Local ID / Peer ID identifiers (where used).

If the device on the other side is behind a NAT, UDP 500 and UDP 4500 traffic must be redirected to that device. FortiGate detects NAT automatically, and the tunnel then operates on port 4500.

Phase 2 does not come up or works only for some networks

Phase 2 selectors must be mirrored on both sides: the local network on the FortiGate side must be the remote network on the other side, and vice versa.

  • Different networks (e.g. 10.12.0.0/24 on the FortiGate and 10.13.0.0/24 on the other side) – Phase 2 fails to establish a connection; the negotiation log displays the message failed to match peer selectors.

  • Different subnets (e.g. 10.12.0.0/24 on the FortiGate and 10.12.0.0/16 on the other side) – with IKEv2, the tunnel is established, but FortiGate restricts the selector to 10.12.0.0/24. Traffic from addresses outside this network (e.g. 10.12.5.10) does not pass through the tunnel. With IKEv1, Phase 2 will not establish.

  • Multiple networks in a single selector – some devices (e.g. Cisco ASA, Juniper) only support one pair of networks per selector. A separate Phase 2 selector must be created for each pair of networks.

Once a new network has been added on one side of the tunnel, it must also be added on the other side, in the static routes and in the firewall policies, and the tunnel must then be restarted (Bring Down, Bring Up).

The tunnel is up, but traffic does not pass

If the tunnel has the status Up but traffic is not passing between the networks, please check the following points in turn.

STEP 1. Connecting the FortiGate to a router in OpenStack

In the CLI, ping the router’s address on the CROSSCONNECT network from the FortiGate’s internal interface address:

execute ping-options source 10.10.0.254
execute ping 10.10.0.1

No response indicates a configuration error in the CROSSCONNECT network – see No communication between FortiGate and the cloud router.

STEP 2. The FortiGate address must not be in use in OpenStack

If a ping to the router works, but not to any instance in the cloud, and packets from the cloud are not reaching the FortiGate, this may be due to an address conflict. With DHCP enabled on the CROSSCONNECT subnet OpenStack assigns the first available address from the pool (e.g. 10.10.0.2 or 10.255.255.2) to the port handling DHCP and metadata. The router in OpenStack then sends traffic destined for that address to that port, rather than to the FortiGate.

You can check which addresses are in use on the network using the following command:

openstack port list --network <CROSSCONNECT_network_name> --long -c "Fixed IP Addresses" -c "Device Owner"

A port with the owner set to network:distributed or network:dhcp is occupied by an address that cannot be used on the FortiGate. In this case, you must set a different, available address from this subnet on the FortiGate’s internal interface and correct the Next Hop in the router’s static routes in OpenStack. New CROSSCONNECT subnets must be created with DHCP disabled.

STEP 3. Routing in OpenStack

The router in the project must have a static route to the network on the other side of the tunnel, specifying the FortiGate address in the CROSSCONNECT network as the Next Hop. You can check the routes under Project -> Network -> Routers -> -> Static Routes.

pic24

You can also check the router’s routes using the command openstack router show <router_name> -c routes.

STEP 4. Routing on FortiGate

In VDOM, there must be a route to the cloud network via the router’s address in the CROSSCONNECT network, and a route to the network on the other side via the tunnel interface (the wizard creates this automatically, together with the Blackhole route). You can check the routes under Network -> Static Routes.

pic25

In the routing table (get router info routing-table all), the cloud network must be visible as a static route (S) via the router in OpenStack:

S       192.168.20.0/24 [10/0] via 10.10.0.1, Internal-vdom-9, [1/0]

If, instead, the message C 192.168.20.0/24 is directly connected, Internal-vdom-9 is displayed, an address from the cloud network has been configured on the internal interface. FortiGate then searches for instances directly on the CROSSCONNECT network, and the other end of the tunnel receives the message ICMP host 192.168.20.10 unreachable. You must remove this address from the interface.

STEP 5. Firewall policies on FortiGate

Under Policy & Objects -> Firewall Policy, there must be policies allowing traffic in both directions between the tunnel interface and the internal interface. In the policies for VPN traffic, the NAT column must be set to Disabled. The By Sequence button displays all policies in the order in which they are checked.

pic26

When NAT is enabled, the cloud instance sees traffic coming from the FortiGate address (e.g. 10.10.0.254), rather than from the network on the other side of the tunnel. Security group rules for that network then cease to apply.

You can use the Policy lookup button to check which policy FortiGate will apply to a particular type of traffic.

STEP 6. Instance security groups

The security group assigned to the instance must allow traffic from the network on the other side of the tunnel (a rule with Remote IP Prefix set to that network, e.g. 10.12.0.0/24). See the section Security Groups.

pic27

STEP 7. The operating system’s firewall

You should check the firewall within the instance (e.g. Windows Defender Firewall, ufw, firewalld, iptables).

STEP 8. The other side of the tunnel

You should check the routing and firewall policies on the device at the other end of the tunnel, and ensure that devices on the local network have a route to the cloud network.

Packet tracking on FortiGate

The diagnose debug flow command shows the paths along which FortiGate forwards packets and whether it drops them. Here is an example for traffic to the instance 192.168.20.10:

diagnose debug flow filter clear
diagnose debug flow filter addr 192.168.20.10
diagnose debug flow show function-name enable
diagnose debug enable
diagnose debug flow trace start 10

Next, you should generate some traffic (e.g. a ping from the network on the other side of the tunnel) and, once this is complete, disable the diagnostics:

diagnose debug disable
diagnose debug flow trace stop
diagnose debug flow filter clear

Correct flow – a packet from the tunnel reaches the router in OpenStack, and the response is sent back to the tunnel:

received a packet(proto=1, 10.12.0.1:6463->192.168.20.10:2048) ... from VPN-Biuro.
find a route: flag=00000000 gw-10.10.0.1 via Internal-vdom-9
Allowed by Policy-2:
received a packet(proto=1, 192.168.20.10:6463->10.12.0.1:0) ... from Internal-vdom-9.
find a route: flag=00000000 gw-77.79.246.39 via VPN-Biuro

The most common results and their causes:

  • Denied by forward policy check (policy 0) – there is no firewall policy for this traffic (STEP 5).

  • SNAT 10.12.0.1->10.10.0.254 – NAT is enabled in the VPN policy (STEP 5).

  • the packet is forwarded to the router (Allowed by Policy), but there is no reply from the internal interface – a missing route in OpenStack, or traffic blocked by a security group or by the instance firewall (STEPS 2, 3, 6, 7).

  • no entries at all – the traffic does not reach the FortiGate. Check the other side of the tunnel or the routing in OpenStack.

The diagnose sniffer packet command is not available for the VDOM administrator account – use diagnose debug flow to monitor traffic.

No communication between the FortiGate and the cloud router

If the FortiGate is not responding to the OpenStack router’s address on the CROSSCONNECT network, please check the following:

  • whether the CROSSCONNECT network has been added as a router interface (ADD INTERFACE) with the correct address,

  • Do the address and subnet mask on the FortiGate’s internal interface belong to the same subnet,

  • Is the router’s address visible in the FortiGate ARP table: get system arp.

If the router’s address does not appear in the ARP table despite the configuration being correct on both sides, please submit a support ticket to Atman – this may indicate a problem with the network connection on Atman’s side.

Connections freeze during larger file transfers

If pinging through the tunnel works, but larger data transfers (e.g. file copying, RDP, database queries) freeze, the cause may be the packet size. An IPsec tunnel adds its own headers, so the maximum packet size within the tunnel is less than 1,500 bytes – for example, for IKEv2 with AES256/SHA256 and the other end behind NAT, it is 1,422 bytes.

The size of packets passing through the tunnel can be checked using a ping command with the ‘do not fragment’ flag set:

  • Linux: ping -M do -s 1394 192.168.20.10

  • Windows: ping -f -l 1394 192.168.20.10

If a smaller ping works but a larger one does not, you should limit the TCP segment size (MSS) in the VPN policies on FortiGate. In the CLI console:

config firewall policy
    edit <ID_polityki>
        set tcp-mss-sender 1350
        set tcp-mss-receiver 1350
    next
end

The command must be executed for both VPN policies (from the cloud to the tunnel and from the tunnel to the cloud). The tunnel policy IDs can be found using the command show firewall policy | grep -f <tunnel_name> – the number is shown on the line edit <ID>.

The tunnel connection drops out from time to time

The most common causes of temporary disconnections:

  • varying key lifetimes (Key Lifetime) in phase 1 or 2 – when keys are rekeyed, the tunnel may disconnect,

  • PFS disabled on one side and enabled on the other – with IKEv2, the problem only becomes apparent during the first Phase 2 key renewal,

  • no traffic in the tunnel – if the other party establishes the tunnel only ‘on demand’, we recommend enabling the Auto-negotiate option in phase 2 and Dead Peer Detection (On Idle) in phase 1,

  • dynamic IP address of the other party (e.g. a connection without a static IP address).

In the event logs, a disconnection caused by Dead Peer Detection is shown as IPsec DPD failed.

What to include in a support request

If the problem has not been resolved, please send an enquiry to servicedesk@atman.pl and include the following:

  • the name of the Atman Cloud service and the name of the tunnel in the VDOM,

  • the public IP address of the other end of the tunnel and the device type (e.g. MikroTik, Cisco ASA, Azure VPN Gateway),

  • the date and time the problem occurred,

  • the parameters of phase 1 and phase 2 on the other side of the tunnel,

  • the output of the commands get vpn ipsec tunnel summary and get router info routing-table all,

  • the addresses of instances with which there is no communication.

Do not include the pre-shared key or the lines containing session keys from the output of the diagnose commands in your report.