Now that we have our functionality, it’s time to make our program more practical and easy to use. For our use case a CLI is the best tool for this!
A CLI or command line interface is a program you use by calling it from a terminal in a directory. If you’re not familiar with how to use the command line very effectively, here’s some resources I would recommend:
Why?
In our case creating a GUI (Graphical User Interface), is mostly going to be slower. Our users are primarily going to be devs and/or tech savy people we expect to know how to use a command line, and a CLI will be faster for what we need.
Here are some of the main reasons we’re creating a CLI at all instead of just making people run the script from the python file:
- It allows you to pass arguments/flags in an easy manner
- It can be invoked from the command line; There’s no need to keep a file in every directory you want to use it in
- It allows you to seperate your more common usage into an easy to use CLI, while allowing an API to exist for more complicated functionality or use cases
- It can be used in other scripts/CI/CD easily since people just need to run it from the command line
So let’s learn about how to setup CLI’s!
Arugment parsing
A CLI allows you to pass in flags/arguments, these flags/arguments are like funciton parameters, but they’re provided by the user before the program starts. This lets users for example choose to run an app in different modes, or pass in information like an output directory for the output of a program etc.
As such the only real “processing” a CLI does besides just running existing code is argument parsing. Argument parsing is basically taking what someone types into the terminal and parsing it into usable data. The most basic way of doing this is sys.argv. sys.argv is the way that information is passed into your program from stdin. For example if I run python hhttpp.py -h, then I can run:
import sys
print(sys.argv) # ['hhttpp.py', '-h']
You can see that everything after python (which is the command) will be added to sys.argv as a list item. We could use this to for example have the first argument be a directory, and the second argument to be a port. For example if I ran python hhttpp.py /site 8338 then sys.argv would be ['hhttpp.py', '/site', '8338']. We could then use these arguments to instantiate a Server object, and then start it with Server.start_server!
Docopt
sys.argv is great for very simple things, however for more flexibility we need something more robust. There are tons of argument parsers, but there are few that are as fast to develop with as docopt. It has a bit of a learning curve, but once you are aware of the patterns it is very fast to write. Docopt will generate your argument parsing automatically based on your usage string.
Usage strings
A usage string is the “help” text that appears for commands. For example a project I wrote called ahd has this usage string:
AHD: Ad-hoc Dispatcher
Usage:
ahd [-h] [-v]
ahd list [-l]
ahd docs [-a] [-o]
ahd config [-e] [-i CONFIG_FILE_PATH]
ahd register <name> [<command>] [<paths>]
ahd <name> [<command>] [<paths>] [-d]
Options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-l, --long Shows all commands in configuration with paths and commands
-a, --api shows the local API docs
-o, --offline shows the local User docs instead of live ones
-e, --export exports the configuration file
-i CONFIG_FILE_PATH, --import CONFIG_FILE_PATH
imports the configuration file
-d, --details prints the details of a command
Each line under the Usage header is a different possible pattern. For example ahd list would use the pattern on line 2, ahd config -e would use line 4. Anything inside square parenthesis ([]) are optional arguments/flags/variables. For everything else you can find details here. Anything with a - is a flag, typically these are used as a boolean where if they’re specified they’re true, and if not they’re false. They also each have a long and short version (i.e. -e and --export are the same flags). Any flag that has text after it (i.e. -i CONFIG_FILE_PATH) allows you to store values for flags, so in this case you can type something like --import="/file/config.yaml".
Let’s put together a quick usage string. Here is the basic format I would always use:
<CLI_name>: <Short Description>
Usage:
<CLI_name> [-h] [-v]
Options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
From here we can start building out a CLI. In our case we can will start by building a usage string for a math CLI (called math) that can be used to do various math operations. So with our template we can start with:
Math: Doing mathy things
Usage:
math [-h] [-v]
Options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
From here we will have the following functionality:
| Command | Arguments/flags | Description |
|---|
| multiply | x and y both ints | Multiplies x * y |
| divide | x and y both ints | Divides x / y |
| idivide | x and y both ints | Divides x / y and rounds down to nearest integer |
| sqrt | x which is an int | Takes square root of x |
| pi | l an optional int for how many decimal places to show pi to (default 6) | Show digits of pi to l digits |
Let’s build out the new usage string:
Math: Doing mathy things
Usage:
math [-h] [-v]
math multiply <x> <y>
math divide <x> <y>
math idivide <x> <y>
math sqrt <x>
math pi [-l]
Options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-l, --length The number of digits of pi you want
Here are the functions we want to call with the CLI:
def multiply(x:int, y:int) -> Union[int, float]:
return x * y
def divide(x:int, y:int) -> Union[int, float]:
return x / y
def idivide(x:int, y:int) -> Union[int, float]:
return x // y
def sqrt(x:int) -> float:
return x**.5
def pi(length:int = 6) -> Union[int, float]:
PI = 3.141592653589793238462643
return float(str(PI)[:length])
Docopt basics
For most other argument parsers you build the parser (Like optparse objects, or argparse objects ), and at the end it generates usage text when people pass in -h or --help.
Docopt does the opposite, you define your usage string and it builds the parser for you. This is handy because it means that for lots of simple CLI’s you can build out the parser in seconds by just writing a usage string. The steps are:
- Define a usage string
- Pass the usage string into a docopt method
- Docopt will parse the usage string, and the values passed in, which will be put into a dictionary you can interface with
With our math example we can do something like this:
from typing import Union
from docopt import docopt
usage = """Math: Doing mathy things
Usage:
math [-h] [-v]
math multiply <x> <y>
math divide <x> <y>
math idivide <x> <y>
math sqrt <x>
math pi [-l]
Options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-l, --length The number of digits of pi you want
"""
def multiply(x:int, y:int) -> Union[int, float]:
return x * y
def divide(x:int, y:int) -> Union[int, float]:
return x / y
def idivide(x:int, y:int) -> Union[int, float]:
return x // y
def sqrt(x:int) -> float:
return x**.5
def pi(length:int = 6) -> Union[int, float]:
PI = 3.141592653589793238462643
return float(str(PI)[:length])
args = docopt(usage, version="0.1.0")
print(args)
So in our case if we called the CLI with math multiply 3 4 we would get:
args == {
'--help': False,
'--length': False,
'--version': False,
'<x>': '3',
'<y>': '4',
'divide': False,
'idivide': False,
'multiply': True,
'pi': False,
'sqrt': False
}
We can call math -h to get our usage string, the same way we would if we built our parser from scratch. From here to implement multiply we could do:
from typing import Union
from docopt import docopt
usage = """Math: Doing mathy things
Usage:
math [-h] [-v]
math multiply <x> <y>
math divide <x> <y>
math idivide <x> <y>
math sqrt <x>
math pi [-l]
Options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-l, --length The number of digits of pi you want
"""
def multiply(x:int, y:int) -> Union[int, float]:
return x * y
def divide(x:int, y:int) -> Union[int, float]:
return x / y
def idivide(x:int, y:int) -> Union[int, float]:
return x // y
def sqrt(x:int) -> float:
return x**.5
def pi(length:int = 6) -> Union[int, float]:
PI = 3.141592653589793238462643
return float(str(PI)[:length])
args = docopt(usage, version="0.1.0")
print(args)
if args["multiply"]:
print(multiply(int(args["<x>"]), int(args["<y>"])))
You will notice that all arguments come in as strings (besides flags which are booleans), so you need to explicitly convert to other data types like int/float. So implementing all our functionality would look something like this:
from typing import Union
from docopt import docopt
usage = """Math: Doing mathy things
Usage:
math [-h] [-v]
math multiply <x> <y>
math divide <x> <y>
math idivide <x> <y>
math sqrt <x>
math pi [-l DIGITS]
Options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-l DIGITS, --length DIGITS
The number of digits of pi you want
"""
def multiply(x:int, y:int) -> Union[int, float]:
return x * y
def divide(x:int, y:int) -> Union[int, float]:
return x / y
def idivide(x:int, y:int) -> Union[int, float]:
return x // y
def sqrt(x:int) -> float:
return x**.5
def pi(length:int = 6) -> Union[int, float]:
PI = 3.141592653589793238462643
return float(str(PI)[:length])
args = docopt(usage, version="0.1.0")
if args["<x>"]:
args["<x>"] = int(args["<x>"])
if args["<y>"]:
args["<y>"] = int(args["<y>"])
if args["--length"]:
args["--length"] = int(args["--length"])
else:
args["--length"] = 6
if args["multiply"]:
print(multiply(args["<x>"], args["<y>"]))
if args["divide"]:
print(divide(args["<x>"], args["<y>"]))
if args["idivide"]:
print(idivide(args["<x>"], args["<y>"]))
if args["sqrt"]:
print(sqrt(args["<x>"]))
if args["pi"]:
print(pi(args["--length"]))
We can test this without installing the cli, if the file is called docopt_testing.py then we can run something like python docopt_testing.py pi --length=15.
When to use docopt and when not to
Docopt is great, and some extra features it grants you without extra config are:
- Another issues docopt solves is ordering, you can
- Invalid states will just print the usage string to the command line so people know they made a mistake
- People can check version they have installed easily with
-v
You do have to learn how to read usage strings, but you should do that anyways since ou need to learn how to read usage strings to use other people’s projects. If you already have these skills you should already know the rules for writing the parser!
Docopt is a great option for argument parsing, but it has it’s drawbacks:
- You must have access to install libraries which is not always possible
- You are introducing a third party library and all it’s bugs
- in very very simple cases it’s more effort to write a usage string than just use
sys.argv
Our setup
For HHTTPP we will be using the following usage string:
hhttpp
Free range artisnal HTTP server
Usage:
hhttpp [-h] [-v] [-p PORT] [-f PROXY_FOLDER]
Options:
-h, --help Show this help message and exit
-v, --version Show program's version number and exit
-p PORT, --port PORT The port to start the server on
-f PROXY_FOLDER, --folder PROXY_FOLDER
Lets you specify a folder to proxy instead of cwd
So in this case we have the option to get the version, or help. From there we have 2 optional positional arguments. -p/--port and f/--folder, the reason these are not optional variables is because with optional variables ORDER MATTERS. Meaning if people wanted to specify a folder they would have to speficy a port, which I didn’t want. In our code we provide sensible defaults, so someone can offer both, or neither of the -p or -f flags, people can also skip the -h, or -v flags completely.
From here the argument parsing looks like this:
# Python Standard Library dependencies
import os # Used to validate paths
import socket # Used to validate ports
from random import randint # Provides a random integer between a range
# Internal Dependencies
from hhttpp import __version__ # Get the current hhttpp version
from hhttpp.classes import Server # Used to instantiate hhttpp Server's
# Third Party Dependencies
from docopt import docopt # Used for argument parsing
usage = """hhttpp
Free range artisnal HTTP server
Usage:
hhttpp [-h] [-v] [-p PORT] [-f PROXY_FOLDER]
Options:
-h, --help Show this help message and exit
-v, --version Show program's version number and exit
-p PORT, --port PORT The port to start the server on
-f PROXY_FOLDER, --folder PROXY_FOLDER
Lets you specify a folder to proxy instead of cwd
"""
def main():
# This will be the primary entrypoint for the CLI
args = docopt(usage, version=__version__) # Will be used in later post to do CLI parsing
# Specify default port and folder if none is provided
port = 8338
folder = "."
# Override defaults if valid alternative was provided
if args["--port"]:
port = int(args["--port"])
if args["--folder"]:
if not os.path.exists(args["--folder"]):
raise ValueError(f"Folder path {args['--folder']} does not exist")
folder = args["--folder"]
# Attempt to assign requested port, retry on failure
valid_port = False
while not valid_port:
# Check if port is open
port_testing_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
destination = ("127.0.0.1", port)
result = port_testing_socket.connect_ex(destination)
if not result: # Port is taken, assign new random one
new_port = randint(1_000,10_000)
print(f"Could not connect to port {port} trying other port {new_port}")
port = new_port
port_testing_socket.close()
else: # Current port is valid
print(f"Valid port found: {port}")
valid_port = True
port_testing_socket.close()
# Start server with specified values!
Server(folder, port=port).start_server()
Other final touches
Now we have a CLI there’s a few extras we can work on to improve our CLI:
- Documenting the CLI and usage in something like a
readme.md, or if it’s really complicated a dedicated site to using the system can be handy (like this project I wrote). Not having a readme or some other documentation makes it really hard for people to use your project (things that seem obvious to you aren’t obvious to everyone). - Documenting API usage for people who want to use it programatically. For example if people want to be able to extend your project, or pass argumetns themselves directly etc. (an example would be the api docs for the same project from earlier)