How to run a flask app on a production server with virtualenv, uwsgi, and nginx

Figuring out how to serve my flask site with virtualenv, uwsgi, and nginx was frustrating. There are plenty of articles on this topic. None of them worked for me on the first try. Here are my notes for Debian Jessie in case someone else is experiencing similar issues.

First make sure that your distro has the latest updates:

1
2
sudo apt-get update
sudo apt-get upgrade

Now install python and virtualenv:

1
2
sudo apt-get install build-essential python-dev python-pip
sudo pip install virtualenv

Make a folder for your website:

1
2
3
sudo mkdir -p /var/www/mysite
sudo chown -R <your user id> /var/www/mysite
cd /var/www/mysite

Setup virtualenv and install flask:

1
2
3
virtualenv .env --no-site-packages
source .env/bin/activate
pip install flask

Place your flask app in this folder. Make sure that your host is set to 0.0.0.0 and that your app is under if name == ‘main‘:. If your app is in a function, uwsgi will not be able to call it.

Now is a good time to test your app with the flask development server to see if everything is working so far. If everything runs smoothly, install nginx and uwsgi:

1
2
deactivate
sudo apt-get install nginx uwsgi uwsgi-plugin-python

Next we must create a socket file for nginx to communicate with uwsgi:

1
2
3
cd /tmp/
touch mysite.sock
sudo chown www-data mysite.sock

By changing the owner of mysite.sock to www-data, nginx will be able to write to the socket. Now all we have to do is add our configuration files for nginx and uwsgi. First delete the default configuration for nginx:

1
2
cd /etc/nginx/sites-available
sudo rm default

Create a new configuration file mysite and add the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
server {
listen 80;
server_tokens off;
server_name www.mysite.com mysite.com;

location / {
include uwsgi_params;
uwsgi_pass unix:/tmp/mysite.sock;
}

location /static {
alias /var/www/mysite/static;
}

## Only requests to our Host are allowed
if ($host !~ ^(mysite.com|www.mysite.com)$ ) {
return 444;
}
}

In order to enable the site, we must link our configuration file to /etc/nginx/sites-enabled/:

1
sudo ln -s /etc/nginx/sites-available/mysite /etc/nginx/sites-enabled/mysite

The process is similar for uwsgi. Create the file /etc/uwsgi/apps-available/mysite.ini and add the following:

1
2
3
4
5
6
7
[uwsgi]
vhost = true
socket = /tmp/mysite.sock
venv = /var/www/mysite/.env
chdir = /var/www/mysite
module = app
callable = app

Module is the name of your python script and callable is the name of your flask instance. So if your flask site was in a file called mysite.py that looked like this:

1
2
3
4
5
6
7
8
9
from flask import Flask
my_app = Flask(__name__)

@my_app.route('/')
def hello_world():
return 'Hello World!'

if __name__ == '__main__':
my_app.run(host='0.0.0.0')

Your mysite.ini file would be:

1
2
module = mysite
callable = my_app

Link the configuration file to the enabled-apps folder:

1
sudo ln -s /etc/uwsgi/apps-available/mysite.ini /etc/uwsgi/apps-enabled/mysite.ini

Finally, restart nginx and uwsgi:

1
2
sudo service nginx restart
sudo service uwsgi restart

Thats it. If you notice any errors in my guide, please feel free to email me. Here are some tips in case you get stuck:

Check that your flask site runs under virtualenv without any errors.
Ensure that you can run your site with just uwsgi from the command line.
If you run uwsgi with sudo you will change the owner of mysite.sock to root and this will create errors for nginx. Make sure that you change the owner back to www-data.
If uwsgi cannot find your app, you probably have an issue with file permissions. In order to serve the site uwsgi must have executable permissions for your python script and your .env folder.
The logs for nginx and uwsgi are /var/log/nginx/error.log and /var/log/uwsgi/app/mysite.log respectively. If nginx is working properly, you will want to look at /var/log/nginx/access.log.

Tree

tree is a recursive directory listing command that produces a depth indented listing of files.

Installation

To install the latest version, use homebrew:

1
brew install tree

Usage

Running tree will produce output like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
.
├── Apps
│ ├── Octave.md
│ ├── README.md
│ ├── Settings.md
│ ├── araxis-merge.jpg
│ ├── beyond-compare.png
│ ├── delta-walker.jpg
│ ├── filemerge.png
│ └── kaleidoscope.png
├── CONTRIBUTING.md
├── Cpp
│ └── README.md
├── Docker
│ └── README.md
├── Git
│ ├── README.md
│ └── gitignore.md
└── Go
└── README.md

5 directories, 14 files

To limit the recursion you can pass an -L flag and specify the maximum depth tree will use when searching.

1
tree -L 1

will output:

1
2
3
4
5
6
7
8
9
.
├── Apps
├── CONTRIBUTING.md
├── Cpp
├── Docker
├── Git
└── Go

5 directories, 1 files

Homebrew

Homebrew calls itself The missing package manager for macOS and is an essential tool for any developer.

Installation

An important dependency before Homebrew can run is the Command Line Tools for Xcode. These include compilers that will allow you to build things from source, if you are missing this it’s available through the App Store > Updates.

To install Homebrew paste the following command (without the $) in your terminal, hit Enter, and follow the steps on the screen:

1
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

One thing we need to do is tell the system to use programs installed by Hombrew (in /usr/local/bin) rather than the OS default if it exists. We do this by adding /usr/local/bin to your $PATH environment variable:

1
$ echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bash_profile

Alternatively, we can also insert /usr/local/bin to the first line of /private/etc/paths and reboot the Mac to change global paths loading order. Admin password may be required if you modify the file.

To be able to use brew you need to start a new terminal session. Do this by opening a new terminal tab with Cmd+T (you should also close the old one), then run the following command to make sure everything is working:

1
$ brew doctor

Using Homebrew

To install a package (or Formula in Homebrew vocabulary) simply type:

1
$ brew install <formula>

To update Homebrew’s directory of formulae, run:

1
$ brew update

Note: I’ve seen that command fail sometimes because of a bug. If that ever happens, run the following (when you have Git installed):

1
2
3
$ cd /usr/local/Homebrew/
$ git fetch origin
$ git reset --hard origin/master

To see if any of your packages need to be updated:

1
$ brew outdated

To update a package:

1
$ brew upgrade <formula>

Homebrew keeps older versions of packages installed, in case you want to roll back. That rarely is necessary, so you can do some cleanup to get rid of those old versions:

1
$ brew cleanup

To see what you have installed (with their version numbers):

1
$ brew list --versions

Homebrew Cask

Homebrew-Cask extends Homebrew and allows you to install large binary files via a command-line tool. Examples of these files is Google Chrome, Dropbox, VLC and Spectacle.

Installation

As of December 2015, Cask comes installed with Homebrew, if you have not installed Homebrew see the Homebrew section.

To see if an app is available on Cask you can search on the official Cask website. You can also search using the following command:

1
$ brew cask search <package>

Quick Look plugins

These plugins adds support for the corresponding file type to Mac Quick Look (In Finder, mark a file and press Space to start Quick Look). The plugins includes features like syntax highlighting, markdown rendering, preview of JSON, patch files, csv, zip files and more.

1
2
3
4
5
6
7
8
9
10
$ brew cask install \
qlcolorcode \
qlstephen \
qlmarkdown \
quicklook-json \
qlprettypatch \
quicklook-csv \
betterzipql \
webpquicklook \
suspicious-package

App Suggestions

Here are some useful apps that are available on Cask.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ brew cask install \
alfred \
android-file-transfer \
asepsis \
appcleaner \
caffeine \
cheatsheet \
docker \
doubletwist \
dropbox \
google-chrome \
google-drive \
google-hangouts \
flux \
latexian \
1password \
pdftk \
spectacle \
sublime-text \
superduper \
totalfinder \
transmission \
valentina-studio \
vlc

Reading and Writing files in Python

Overview

When you’re working with Python, you don’t need to import a library in order to read and write files. It’s handled natively in the language, albeit in a unique manner.

The first thing you’ll need to do is use Python’s built-in open function to get a file object.

The open function opens a file. It’s simple.

When you use the open function, it returns something called a file object. File objects contain methods and attributes that can be used to collect information about the file you opened. They can also be used to manipulate said file.

For example, the mode attribute of a file object tells you which mode a file was opened in. And the name attribute tells you the name of the file that the file object has opened.

You must understand that a file and file object are two wholly separate – yet related – things.

File Types

What you may know as a file is slightly different in Python.

In Windows, for example, a file can be any item manipulated, edited or created by the user/OS. That means files can be images, text documents, executables, and much more. Most files are organized by keeping them in individual folders.

In Python, a file is categorized as either text or binary, and the difference between the two file types is important.

Text files are structured as a sequence of lines, where each line includes a sequence of characters. This is what you know as code or syntax.

Each line is terminated with a special character, called the EOL or End of Line character. There are several types, but the most common is the comma {,} or newline character. It ends the current line and tells the interpreter a new one has begun.

A backslash character can also be used, and it tells the interpreter that the next character – following the slash – should be treated as a new line. This character is useful when you don’t want to start a new line in the text itself but in the code.

A binary file is any type of file that is not a text file. Because of their nature, binary files can only be processed by an application that know or understand the file’s structure. In other words, they must be applications that can read and interpret binary.

Open ( ) Function

In order to open a file for writing or use in Python, you must rely on the built-in open () function.

As explained above, open ( ) will return a file object, so it is most commonly used with two arguments.

An argument is nothing more than a value that has been provided to a function, which is relayed when you call it. So, for instance, if we declare the name of a file as “Test File,” that name would be considered an argument.

The syntax to open a file object in Python is:

file_object = open(“filename”, “mode”) where file_object is the variable to add the file object.
The second argument you see – mode – tells the interpreter and developer which way the file will be used.

Mode

Including a mode argument is optional because a default value of ‘r’ will be assumed if it is omitted. The ‘r’ value stands for read mode, which is just one of many.

The modes are:

‘r’ – Read mode which is used when the file is only being read
‘w’ – Write mode which is used to edit and write new information to the file (any existing files with the same name will be erased when this mode is activated)
‘a’ – Appending mode, which is used to add new data to the end of the file; that is new information is automatically amended to the end
‘r+’ – Special read and write mode, which is used to handle both actions when working with a file
So, let’s take a look at a quick example.

1
2
F = open(“workfile”,”w”) 
Print f

This snippet opens the file named “workfile” in writing mode so that we can make changes to it. The current information stored within the file is also displayed – or printed – for us to view.

Once this has been done, you can move on to call the objects functions. The two most common functions are read and write.

Create a text file

To get more familiar with text files in Python, let’s create our own and do some additional exercises.

Using a simple text editor, let’s create a file. You can name it anything you like, and it’s better to use something you’ll identify with.

For the purpose of this tutorial, however, we are going to call it “testfile.txt”.

Just create the file and leave it blank.

To manipulate the file, write the following in your Python environment (you can copy and paste if you’d like):

1
2
3
4
5
6
7
8
file = open(“testfile.txt”,”w”) 

file.write(“Hello World”)
file.write(“This is our new text file”)
file.write(“and this is another line.”)
file.write(“Why? Because we can.”)

file.close()

Naturally, if you open the text file – or look at it – using Python you will see only the text we told the interpreter to add.

1
2
3
4
5
$ cat testfile.txt 
Hello World
This is our new text file
and this is another line.
Why? Because we can.

Reading a Text File in Python

There are actually a number of ways to read a text file in Python, not just one.

If you need to extract a string that contains all characters in the file, you can use the following method:

1
file.read()

The full code to work with this method will look something like this:

1
2
file = open(“testfile.text”, “r”) 
print file.read()

The output of that command will display all the text inside the file, the same text we told the interpreter to add earlier. There’s no need to write it all out again, but if you must know, everything will be shown except for the “$ cat testfile.txt” line.

Another way to read a file is to call a certain number of characters.

For example, with the following code the interpreter will read the first five characters of stored data and return it as a string:

1
2
3
file = open(“testfile.txt”, “r”)

print file.read(5)

Notice how we’re using the same file.read() method, only this time we specify the number of characters to process?

The output for this will look like:

1
Hello

If you want to read a file line by line – as opposed to pulling the content of the entire file at once – then you use the readline() function.

Why would you use something like this?

Let’s say you only want to see the first line of the file – or the third. You would execute the readline() function as many times as possible to get the data you were looking for.

Each time you run the method, it will return a string of characters that contains a single line of information from the file.

1
2
file = open(“testfile.txt”, “r”) 
print file.readline():

This would return the first line of the file, like so:

1
Hello World

If we wanted to return only the third line in the file, we would use this:

1
2
file = open(“testfile.txt”, “r”) 
print file.readline(3):

But what if we wanted to return every line in the file, properly separated? You would use the same function, only in a new form. This is called the file.readlines() function.

1
2
file = open(“testfile.txt”, “r”) 
print file.readlines()

The output you would get from this is:

1
[‘Hello World’, ‘This is our new text file’, ‘and this is another line.’, ‘Why? Because we can.’]

Notice how each line is separated accordingly? Note that this is not the ideal way to show users the content in a file. But it’s great when you want to collect information quickly for personal use during development or recall.

Looping over a file object

When you want to read – or return – all the lines from a file in a more memory efficient, and fast manner, you can use the loop over method. The advantage to using this method is that the related code is both simple and easy to read.

1
2
3
file = open(“testfile.txt”, “r”) 
for line in file:
print line,

This will return:

1
2
3
4
Hello World 
This is our new text file
and this is another line.
Why? Because we can.

See how much simpler that is than the previous methods?

Using the File Write Method

One thing you’ll notice about the file write method is that it only requires a single parameter, which is the string you want to be written.

This method is used to add information or content to an existing file. To start a new line after you write data to the file, you can add an EOL character.

1
2
3
4
5
6
file = open(“testfile.txt”, “w”)

file.write(“This is a test”)
file.write(“To add more lines.”)

file.close()

Obviously, this will amend our current file to include the two new lines of text. There’s no need to show output.

Closing a File

When you’re done working, you can use the fh.close() command to end things. What this does is close the file completely, terminating resources in use, in turn freeing them up for the system to deploy elsewhere.

It’s important to understand that when you use the fh.close() method, any further attempts to use the file object will fail.

Notice how we have used this in several of our examples to end interaction with a file? This is good practice.

File Handling in the Real World

To help you better understand some of the methods discussed here, we’re going to offer a few examples of them being used in the real world. Feel free to copy the code and try it out for yourself in a Python interpreter (make sure you have any named files created and accessible first).

Opening a text file:

fh = open(“hello.txt”, “r”) 

Reading a text file:

Fh = open(“hello.txt”, “r”) 
print fh.read() 

To read a text file one line at a time:

fh = open(“hello.text”, “r”) 
print fh.readline() 

To read a list of lines in a text file:

fh = open(“hello.txt”, “r”) 
print fh.readlines() 

To write new content or text to a file:

fh = open(“hello.txt”, “w”) 

fh.write(“Put the text you want to add here”) 
fh.write(“and more lines if need be.”) 

fh.close() 

You can also use this to write multiple lines to a file at once:

fh = open(“hello.txt”,”w”) 
lines_of_text = [“One line of text here”, “and another line here”, “and yet another here”, “and so on and so forth”] 
fh.writelines(lines_of_text) 
fh.close() 

To append a file:

fh = open(“hello.txt”, “a”) 
fh.write(“We Meet Again World”) 
fh.close 

To close a file completely when you are done:

fh = open(“hello.txt”, “r”) 
print fh.read() 
fh.close() 

With Statement

You can also work with file objects using the with statement. It is designed to provide much cleaner syntax and exceptions handling when you are working with code. That explains why it’s good practice to use the with statement where applicable.

One bonus of using this method is that any files opened will be closed automatically after you are done. This leaves less to worry about during cleanup.

To use the with statement to open a file:

with open(“filename”) as file: 

Now that you understand how to call this statement, let’s take a look at a few examples.

with open(“testfile.txt”) as file:  
data = file.read() 
do something with data 

You can also call upon other methods while using this statement. For instance, you can do something like loop over a file object:

with open(“testfile.txt”) as f: 
for line in f: 
print line, 

You’ll also notice that in the above example we didn’t use the “file.close()” method because the with statement will automatically call that for us upon execution. It really makes things a lot easier, doesn’t it?

Using the With Statement in the Real World

To better understand the with statement, let’s take a look at some real world examples just like we did with the file handling functions.

To write to a file using the with statement:

with open(“hello.txt”, “w”) as f: 
f.write(“Hello World”) 

To read a file line by line, output into a list:

with open(“hello.txt”) as f: 
data = f.readlines() 

This will take all of the text or content from the “hello.txt” file and store it into a string called “data”.

Splitting Lines in a Text File

As a final example, let’s explore a unique function that allows you to split the lines taken from a text file. What this is designed to do, is split the string contained in variable data whenever the interpreter encounters a space character.

But just because we are going to use it to split lines after a space character, doesn’t mean that’s the only way. You can actually split your text using any character you wish - such as a colon, for instance.

The code to do this (also using a with statement) is:

with open(“hello.text”, “r”) as f:
data = f.readlines()

for line in data:
words = line.split()
print words

If you wanted to use a colon instead of a space to split your text, you would simply change line.split() to line.split(“:”).

The output for this will be:

[“hello”, “world”, “how”, “are”, “you”, “today?”]
[“today”, “is”, “Saturday”]

The reason the words are presented in this manner is because they are stored – and returned – as an array. Be sure to remember this when working with the split function.

Handling simple Python/Json requests

Using the open weather map api as an example we can use the following steps in python to request some data from the api, which will return a json object. This is then parsed into a readable object that we can retrieve the data from that is of use to us.

1
2
3
4
5
6
7
8
9
import requests
import json

r = requests.get('http://api.openweathermap.org/data/2.5/weather?q=London') #return json object
json_data = json.loads(r.text) #convert object to dictionary

print json_data #will print the entire dictionary of objects

print json_data['sys']['country'] #parse the data you require from the object

It really is that simple!

Install and Activate a Python Virtual Environment

Installing the virtualenv package will isolate our Python project from the system’s Python environment.

We can easily install this using home-brew:

1
brew install python-virtualenv

Now create a projects folder in our your home directory and then create a virtual environment within this folder:

1
2
3
mkdir ~/projects
cd ~/projects
virtualenv --no-site-packages venv

This creates a directory called venv within the projects directory. It installs some Python utilities within this folder and creates a directory structure to install additional tools.

Now activate the virtual environment:

1
source venv/bin/activate

The command prompt will change to reflect the fact that we are operating in a virtual environment. If you need to exit the virtual environment, you can type deactivate at any time.

Plotting graphs in Python

Plotting graphs in python is quite straightforward. You need to make sure that scipy, numpy and matplotlib libraries are installed. Once installed we can then create a graph quite simply by using the following commands.

1
2
3
4
import matplotlib.pyplot as plt 
plt.plot([1,4,5,7],[3,4,4,5])
plt.ylabel('some numbers')
plt.savefig('graph.png')

Will produce the following plot:

A simple python graph plot

You can do other things with the graph modules, below is a small python script I wrote to calculate the chances of rolling the numbers 2 thru 12 with two dice. We can add axis labels, titles, change the graph style and much more.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import pylab
import random

sampleSize = 1000000

twoDice = []
for i in range(sampleSize):
newValue = random.randint(1,6) + random.randint(1,6)
twoDice.append(newValue)

print("Results for throwing a single die", sampleSize, "times.")
print("Mean of the sample =", pylab.mean(twoDice))
print("Median of the sample =", pylab.median(twoDice))
print("Standard deviation of the sample =", pylab.std(twoDice))

pylab.hist(twoDice, bins = pylab.arange(1.5,12.6,1.0))
pylab.xlabel('Value')
pylab.ylabel('Count')
pylab.savefig('twoDice.png')

Average dice rolls python plot

Creating Github repo's remotely from the Command Line

To do this we have to use the gitHub API as github provides no shell access. So using a simpe curl statement we can trigger the api to insert a new repo.

1
curl -u 'USER' https://api.github.com/user/repos -d '{"name":"REPO"}'

Remember replace USER with your username and REPO with your repository/application name!

Then we follow the usual github routine of creating a new or pushing a local repo to our newly created container on github.

Create a new repository on the command line

1
2
3
4
5
6
touch README.md
git init
git add README.md
git commit -m "first commit"
git remote add origin https://github.com/USER/NAMEOFREPOSITORY
git push -u origin master

Push an existing repository from the command line

1
2
git remote add origin https://github.com/USER/NAMEOFREPOSITORY
git push -u origin master

Python Dictionaries

To create a dictionary in python:

1
myDict = {'Name':'John', 'Age':32, 24:'Python', 34:23......etc }

The key is to the left of the colon, the value to the Right. Dictionaries are mutable and can hold any objects. Keys have to be of an immutable type.

To access the values within the dictionary you have to use the respective key.

1
myDict['Age']

will return 32 from the example above.

1
myDict[34]

will return 23

If you request a key that isn’t in the dictionary you will return an error. Therefore it is a good idea to use the .get method.

1
myDict.get('Name')

will return ‘John’

1
myDict.get(54)

will return Null instead of an error.

You can also add a default return value

1
myDict.get(key, default=None)

default – This is the Value to be returned in case key does not exist.

Crontab How To

An experienced Linux sysadmin knows the importance of running the routine maintenance jobs in the background automatically.

Linux Cron utility is an effective way to schedule a routine background job at a specific time and/or day on an on-going basis.

This article is part of the on-going Productivity Tips For Geeks series. In this article, let us review 15 awesome examples of crontab job scheduling.

Linux Crontab Format

MIN HOUR DOM MON DOW CMD

Crontab Fields and Allowed Ranges (Linux Crontab Syntax)

Field Description Allowed Value
MIN Minute field 0 to 59
HOUR Hour field 0 to 23
DOM Day of Month 1-31
MON Month field 1-12
DOW Day Of Week 0-6
CMD Command Any command to be executed

1. Scheduling a Job For a Specific Time

The basic usage of cron is to execute a job in a specific time as shown below. This will execute the Full backup shell script (full-backup) on 10th June 08:30 AM.

Please note that the time field uses 24 hours format. So, for 8 AM use 8, and for 8 PM use 20.

30 08 10 06 * /home/username/full-backup

30 – 30th Minute
08 – 08 AM
10 – 10th Day
06 – 6th Month (June)
* – Every day of the week

2. Schedule a Job For More Than One Instance (e.g. Twice a Day)

The following script take a incremental backup twice a day every day.

This example executes the specified incremental backup shell script (incremental-backup) at 11:00 and 16:00 on every day. The comma separated value in a field specifies that the command needs to be executed in all the mentioned time.

00 11,16 * * * /home/username/bin/incremental-backup

00 – 0th Minute (Top of the hour)
11,16 – 11 AM and 4 PM
* – Every day
* – Every month
* – Every day of the week

3. Schedule a Job for Specific Range of Time (e.g. Only on Weekdays)

If you wanted a job to be scheduled for every hour with in a specific range of time then use the following.

Cron Job everyday during working hours

This example checks the status of the database everyday (including weekends) during the working hours 9 a.m – 6 p.m

00 09-18 * * * /home/username/bin/check-db-status

00 – 0th Minute (Top of the hour)
09-18 – 9 am, 10 am,11 am, 12 am, 1 pm, 2 pm, 3 pm, 4 pm, 5 pm, 6 pm
* – Every day
* – Every month
* – Every day of the week

Cron Job every weekday during working hours

This example checks the status of the database every weekday (i.e excluding Sat and Sun) during the working hours 9 a.m – 6 p.m.

00 09-18 * * 1-5 /home/username/bin/check-db-status

00 – 0th Minute (Top of the hour)
09-18 – 9 am, 10 am,11 am, 12 am, 1 pm, 2 pm, 3 pm, 4 pm, 5 pm, 6 pm
* – Every day
* – Every month
1-5 -Mon, Tue, Wed, Thu and Fri (Every Weekday)

4. How to View Crontab Entries?

View Current Logged-In User’s Crontab entries

To view your crontab entries type crontab -l from your unix account as shown below.

1
2
3
4
username@dev-db$ crontab -l

@yearly /home/username/annual-maintenance
*/10 * * * * /home/username/check-disk-space

[Note: This displays crontab of the current logged in user]

View Root Crontab entries

Login as root user (su – root) and do crontab -l as shown below.

1
2
3
root@dev-db# crontab -l

no crontab for root
Crontab HowTo: View Other Linux User’s Crontabs entries

To view crontab entries of other Linux users, login to root and use -u {username} -l as shown below.

1
2
3
4
root@dev-db# crontab -u otheruser -l

@monthly /home/otheruser/monthly-backup
00 09-18 * * * /home/otheruser/check-db-status

5. How to Edit Crontab Entries?

Edit Current Logged-In User’s Crontab entries

To edit crontab entries, use crontab -e as shown below. By default this will edit the current logged-in users crontab.

1
2
3
4
5
6
username@dev-db$ crontab -e

@yearly /home/username/centos/bin/annual-maintenance
*/10 * * * * /home/username/debian/bin/check-disk-space
~
"/tmp/crontab.XXXXyjWkHw" 2L, 83C

[Note: This will open the crontab file in Vim editor for editing.
Please note cron created a temporary /tmp/crontab.XX… ]
When you save the above temporary file with :wq, it will save the crontab and display the following message indicating the crontab is successfully modified.

1
2
3
~
"crontab.XXXXyjWkHw" 2L, 83C written
crontab: installing new crontab
Edit Root Crontab entries

Login as root user (su – root) and do crontab -e as shown below.

1
root@dev-db# crontab -e
Edit Other Linux User’s Crontab File entries

To edit crontab entries of other Linux users, login to root and use -u {username} -e as shown below.

1
2
3
4
5
6
7
8
root@dev-db# crontab -u otheruser -e

@monthly /home/otheruser/fedora/bin/monthly-backup
00 09-18 * * * /home/otheruser/ubuntu/bin/check-db-status
~
~
~
"/tmp/crontab.XXXXyjWkHw" 2L, 83C

6. Schedule a Job for Every Minute Using Cron.

Ideally you may not have a requirement to schedule a job every minute. But understanding this example will will help you understand the other examples mentioned below in this article.

* * * * * CMD

The * means all the possible unit — i.e every minute of every hour through out the year. More than using this * directly, you will find it very useful in the following cases.

When you specify */5 in minute field means every 5 minutes.
When you specify 0-10/2 in minute field mean every 2 minutes in the first 10 minute.

Thus the above convention can be used for all the other 4 fields.

7. Schedule a Background Cron Job For Every 10 Minutes.

Use the following, if you want to check the disk space every 10 minutes.

*/10 * * * * /home/username/check-disk-space

It executes the specified command check-disk-space every 10 minutes through out the year. But you may have a requirement of executing the command only during office hours or vice versa. The above examples shows how to do those things.

Instead of specifying values in the 5 fields, we can specify it using a single keyword as mentioned below.

There are special cases in which instead of the above 5 fields you can use @ followed by a keyword — such as reboot, midnight, yearly, hourly.

Cron special keywords and their meanings

Keyword Equivalent
@yearly 0 0 1 1 *
@daily 0 0 * * *
@hourly 0 * * * *
@reboot Run at startup.

8. Schedule a Job For First Minute of Every Year using @yearly

If you want a job to be executed on the first minute of every year, then you can use the @yearly cron keyword as shown below.

This will execute the system annual maintenance using annual-maintenance shell script at 00:00 on Jan 1st for every year.

@yearly /home/username/red-hat/bin/annual-maintenance

9. Schedule a Cron Job Beginning of Every Month using @monthly

It is as similar as the @yearly as above. But executes the command monthly once using @monthly cron keyword.

This will execute the shell script tape-backup at 00:00 on 1st of every month.

@monthly /home/username/suse/bin/tape-backup

10. Schedule a Background Job Every Day using @daily

Using the @daily cron keyword, this will do a daily log file cleanup using cleanup-logs shell scriptat 00:00 on every day.

@daily /home/username/arch-linux/bin/cleanup-logs "day started"

11. How to Execute a Linux Command After Every Reboot using @reboot?

Using the @reboot cron keyword, this will execute the specified command once after the machine got booted every time.

@reboot CMD

12. How to Disable/Redirect the Crontab Mail Output using MAIL keyword?

By default crontab sends the job output to the user who scheduled the job. If you want to redirect the output to a specific user, add or update the MAIL variable in the crontab as shown below.

1
2
3
4
5
6
username@dev-db$ crontab -l

MAIL="username"

@yearly /home/username/annual-maintenance
*/10 * * * * /home/username/check-disk-space

[Note: Crontab of the current logged in user with MAIL variable]

If you wanted the mail not to be sent to anywhere, i.e to stop the crontab output to be emailed, add or update the MAIL variable in the crontab as shown below.

MAIL=""

13. How to Execute a Linux Cron Jobs Every Second Using Crontab.

You cannot schedule an every-second cronjob. Because in cron the minimum unit you can specify is minute. In a typical scenario, there is no reason for most of us to run any job every second in the system.

14. Specify PATH Variable in the Crontab

All the above examples we specified absolute path of the Linux command or the shell-script that needs to be executed.

For example, instead of specifying /home/username/tape-backup, if you want to just specify tape-backup, then add the path /home/username to the PATH variable in the crontab as shown below.

1
2
3
4
5
6
username@dev-db$ crontab -l

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/home/username

@yearly annual-maintenance
*/10 * * * * check-disk-space

[Note: Crontab of the current logged in user with PATH variable]

15. Installing Crontab From a Cron File

Instead of directly editing the crontab file, you can also add all the entries to a cron-file first. Once you have all thoese entries in the file, you can upload or install them to the cron as shown below.

1
2
3
4
5
6
7
8
9
10
11
12
13
username@dev-db$ crontab -l

no crontab for username

$ cat cron-file.txt
@yearly /home/username/annual-maintenance
*/10 * * * * /home/username/check-disk-space

username@dev-db$ crontab cron-file.txt

username@dev-db$ crontab -l
@yearly /home/username/annual-maintenance
*/10 * * * * /home/username/check-disk-space

Note: This will install the cron-file.txt to your crontab, which will also remove your old cron entries. So, please be careful while uploading cron entries from a cron-file.txt.

^