Home / OCI

OCITroubleshootingIntermediate

OCI File Storage NFS mount hangs or fails with access denied: troubleshooting guide

Fix OCI File Storage mounts that hang, time out, return access denied or write as nobody, by checking security rules, export options, mount target IPs and the client.

Deepak Bhardwaj7 min read

Applies toOCI File Storage service (NFSv3 mount targets), Oracle Linux 8/9, RHEL-compatible clients, oci-fss-utils for in-transit encryption

A mount against an OCI File Storage mount target either hangs, fails with access denied by server, or succeeds but root can't write. Almost every case comes down to four things: the network path to the mount target, the export options, the export path, or the client itself. Work through them in that order.

Symptom to cause

Symptom Likely cause Fix
mount hangs, then Connection timed out Security list or NSG missing ports 111 / 2048-2050, or no route Add the ingress and egress rules below
mount.nfs: access denied by server while mounting Client IP not in the export's source CIDR, or requirePrivilegedSourcePort is true and the client uses a high port Correct export options
mount.nfs: No such file or directory Wrong export path (it isn't the file system name) Read the path from oci fs export list
Mount works, root gets Permission denied or files owned by nobody identitySquash set to ROOT or ALL Change squash or use a non-root UID that owns the directory
Export is read-only access is READ_ONLY for the matching source Change access to READ_WRITE for that CIDR
Mount works from one subnet, not another First-matching export option entry applies a different rule Reorder or tighten the export option list
Stale file handle after a change Export or file system was deleted and recreated Unmount with -l, remount

Gather the facts

Set a few variables and pull the mount target, its IP and the export paths.

C=ocid1.compartment.oc1..xxxx
AD="Uocm:EU-FRANKFURT-1-AD-1"

# Mount targets in the AD
oci fs mount-target list --compartment-id $C --availability-domain "$AD" \
  --query 'data[].{name:"display-name",id:id,state:"lifecycle-state",exportSet:"export-set-id",subnet:"subnet-id"}' \
  --output table

# Private IP of the mount target
MT=ocid1.mounttarget.oc1..xxxx
PIP=$(oci fs mount-target get --mount-target-id $MT --query 'data."private-ip-ids"[0]' --raw-output)
oci network private-ip get --private-ip-id $PIP --query 'data."ip-address"' --raw-output

# Exports and their paths
EXPORT_SET=$(oci fs mount-target get --mount-target-id $MT --query 'data."export-set-id"' --raw-output)
oci fs export list --export-set-id $EXPORT_SET \
  --query 'data[].{path:path,fs:"file-system-id",id:id,state:"lifecycle-state"}' --output table

Note

The export path (for example /shared) is set when the export is created. It isn't the file system's display name, and it is case-sensitive.

Check 1: network path and security rules

From the client, test the ports before you touch NFS:

MT_IP=10.0.2.25
for p in 111 2048 2049 2050; do timeout 3 bash -c "</dev/tcp/$MT_IP/$p" && echo "tcp $p open" || echo "tcp $p BLOCKED"; done
rpcinfo -p $MT_IP

If any port is blocked, check the security list or NSG on both the mount target subnet and the client subnet. Oracle's documented rules for NFS are:

Direction Where Protocol Destination ports
Ingress Mount target, from client CIDR TCP 111, 2048, 2049, 2050
Ingress Mount target, from client CIDR UDP 111, 2048
Egress Client, to mount target CIDR TCP 111, 2048, 2049, 2050
Egress Client, to mount target CIDR UDP 111, 2048
Ingress Mount target, for in-transit encryption TCP 2051

Stateful rules are the simplest choice. If you use stateless rules you also need the matching return-traffic rules in the other direction.

Look at the NSGs attached to the mount target:

oci fs mount-target get --mount-target-id $MT --query 'data."nsg-ids"'
oci network nsg rules list --nsg-id ocid1.networksecuritygroup.oc1..xxxx --all \
  --query 'data[].{dir:direction,proto:protocol,src:source,dst:destination,tcp:"tcp-options"}' --output table

If the client is in another VCN or on-premises, also confirm the route tables and DRG attachments carry the mount target subnet, and that any firewall in the path allows the same ports. See OCI VCN routing for the route-table checks.

Check 2: export options

Export options are an ordered list. The first entry whose source matches the client IP wins, and if nothing matches the client is denied.

EXPORT=ocid1.export.oc1..xxxx
oci fs export get --export-id $EXPORT --query 'data."export-options"'

Fields to read:

Field What it does
source Client CIDR this entry applies to
access READ_WRITE or READ_ONLY
identitySquash NONE, ROOT or ALL; squashed users map to anonymousUid/anonymousGid
requirePrivilegedSourcePort If true, the client must connect from a port below 1024

Update the list. This replaces the whole list, so include every entry you want to keep:

oci fs export update --export-id $EXPORT --export-options '[
  {"source":"10.0.10.0/24","access":"READ_WRITE","identitySquash":"NONE","requirePrivilegedSourcePort":true},
  {"source":"10.0.20.0/24","access":"READ_ONLY","identitySquash":"ROOT","anonymousUid":65534,"anonymousGid":65534,"requirePrivilegedSourcePort":true}
]'

Warning

oci fs export update --export-options overwrites every entry. Save the current output first. Dropping an entry immediately blocks every client in that CIDR, which takes down running applications that use the share.

Caution

0.0.0.0/0 with READ_WRITE and identitySquash: NONE lets any host that can reach the mount target act as root on the share. Keep source as narrow as your client subnets.

Check 3: the client

# Packages
rpm -q nfs-utils || sudo dnf install -y nfs-utils

# Mount with NFSv3, as in Oracle's examples
sudo mkdir -p /mnt/shared
sudo mount -t nfs -o nfsvers=3 -v 10.0.2.25:/shared /mnt/shared

# What's mounted and with what options
nfsstat -m

The -v output tells you which stage fails: portmapper (port 111), mountd (2048), or nfs (2049).

For /etc/fstab, add _netdev so boot doesn't block when the network isn't up yet:

10.0.2.25:/shared  /mnt/shared  nfs  defaults,nfsvers=3,_netdev,nofail  0 0

Check with sudo mount -a before you reboot. A bad fstab line plus no nofail can leave an instance stuck at boot; recovery then goes through the serial console.

In-transit encryption

Encrypted mounts use the oci-fss-utils package on Oracle Linux, which runs a local TLS proxy and talks to the mount target on TCP 2051:

sudo dnf install -y oci-fss-utils
sudo mount -t oci-fss 10.0.2.25:/shared /mnt/shared

If a normal NFS mount works and the encrypted one hangs, port 2051 is the first thing to check. Package availability and supported OS versions change; check the current File Storage in-transit encryption documentation for your distribution.

Check 4: permissions after a successful mount

id
ls -ldn /mnt/shared
touch /mnt/shared/.writetest && echo ok
  • Files show owner 65534 or nobody: identity squash is mapping your user. Either set identitySquash to NONE for that CIDR, or have a non-root UID own the directory.
  • UIDs differ between clients: NFSv3 uses numeric UIDs, so appuser must have the same UID and GID on every client.
  • A new file system has a root-owned top directory. Mount it once from a trusted admin host with squash NONE and chown it to the application user.

Hung mounts on a running server

If the mount target becomes unreachable, processes that touch the mount hang in D state.

ps -eo pid,stat,cmd | awk '$2 ~ /D/'
sudo umount -f -l /mnt/shared

Fix the network or export, then remount. Make sure the mount target itself is ACTIVE, and check the File Storage metrics in the oci_filestorage namespace in Metrics Explorer for throughput or latency problems (metric names vary; list them with oci monitoring metric list --namespace oci_filestorage).

Prevent it next time

  • Put mount targets in a dedicated subnet with an NSG that holds exactly the ports above, and attach a matching client NSG to the instances.
  • Keep export options in Terraform or a saved JSON file so updates are reviewed, not typed by hand.
  • Always use _netdev,nofail in /etc/fstab for NFS.
  • Standardise application UIDs and GIDs across all clients.
  • Alarm on the instance side: a simple cron check that stats the mount with a timeout and posts a custom metric, alerting through your Monitoring alarms.