Computer Science Atlas
Code Review

Python 3 Examples: Create a Directory with Parents (Like mkdir -p)

February 8, 2021
 

mkdir -p

Let's say you want to create a directory /tmp/my/new/dir/, but the intermediate parent directories /tmp/my/ and /tmp/my/new/ don't already exist. On the Linux command line, you would use:

Bash
Copy
mkdir -p /tmp/my/new/dir
$
mkdir -p /tmp/my/new/dir

to create the new directory, as well as the intermediate parent directories all at once.

Using pathlib (Python 3.5 and up)

The Python 3.5+ equivalent to the mkdir -p command is:

Copy
from pathlib import Path

Path( '/tmp/my/new/dir' ).mkdir( parents=True, exist_ok=True )
1
2
3
from pathlib import Path

Path( '/tmp/my/new/dir' ).mkdir( parents=True, exist_ok=True )

The parents=True tells the mkdir command to also create any intermediate parent directories that don't already exist.

exist_ok=True replicates another feature of mkdir -p, where the command does nothing and does not raise an error if the directory already exists.

If you also want to set the directory permissions mode as you create it, you can call the function simply with positional arguments:

Copy
from pathlib import Path

Path( '/tmp/my/new/dir' ).mkdir( 0o755, True, True )
1
2
3
from pathlib import Path

Path( '/tmp/my/new/dir' ).mkdir( 0o755, True, True )

The 0o at the front of 0o755 means that we intend 755 to be interpreted as an octal number — the format that is commonly used with the chmod command on Linux and other Unix-like systems.

Using os.makedirs

Python 3.4.1 and up

If you are using Python 3.4.1 or higher, you can use:

Copy
import os

os.makedirs( '/tmp/my/new/dir', exist_ok=True )
1
2
3
import os

os.makedirs( '/tmp/my/new/dir', exist_ok=True )

os.makedirs always automatically creates any intermediate parent directories that don't already exist, and the exist_ok=True argument tells makedirs not to raise an error if the /tmp/my/new/dir/ directory already exists.

As with pathlib's mkdir, if you also want to set the directory permissions mode as you create it, you can call the function simply with positional arguments:

Copy
import os

os.makedirs( '/tmp/my/new/dir', 0o755, True )
1
2
3
import os

os.makedirs( '/tmp/my/new/dir', 0o755, True )

Lower than Python 3.4.1

Python versions older than 3.4.1 have an os.makedirs function that creates missing intermediate parent directories, but the function either lacks an exist_ok parameter (lower than Python 3.2) or it is not thread-safe (Python 3.2 to before Python 3.4.1).

For these older versions of Python 3, you can manually implement the error handling logic behind the exist_ok parameter:

Copy
import os

def mkdirp( path ):
    try:
        os.makedirs( path )
    except OSError:
        if not os.path.isdir( path ):
            raise

mkdirp( '/tmp/my/new/dir' )
1
2
3
4
5
6
7
8
9
10
import os

def mkdirp( path ):
    try:
        os.makedirs( path )
    except OSError:
        if not os.path.isdir( path ):
            raise

mkdirp( '/tmp/my/new/dir' )

When os.makedirs fails because the directory already exists, it will raise an OSError. The code above excepts all OSErrors because there is no way to be certain exactly what error the operating system will raise if the directory already exists. Our code then re-raises the error (line 8) only if the directory we want to create does not exist. If the directory does exist, then we ignore whatever may have caused the OSError (the error was probably raised because the directory already exists).

You may wonder why the code doesn't simply check if the directory exists before attempting to create it. Even if we check before, we still need the try-except because the following race condition may happen:

  1. We check if the directory exists. The operating system says it does not exist.
  2. Another program creates the directory before our program goes to the next line.
  3. We then attempt to create the directory using os.makedirs. This raises an error because the directory now exists, even though our previous check told us it didn't.

Since the first check doesn't prevent the error in cases like this, we simply omit it altogether and just check if the directory exists only when os.makedirs actually raises an error.

References