Working with text in Python

Most of what a script does is move text around: take a line apart, keep the field you want, put the fields back in a different order, print the result. Every program below is that job, six to eight lines long, with a few gaps to fill.

What you'll be able to do

• Pick a slice out of a string and know what the numbers mean.
• Take a record apart with split and read one field out of it.
• Put fields back together with join, the right way round.
• Loop over a list of lines and build a second list from them.

Step 1: Three slices out of one string

A slice is [start:stop], and it stops before the position you name. Three gaps, three slices, one printed line.

code[:3] is INV. code[4:8] starts at position four and stops before eight, which gives the year. code[-4:] counts four back from the end. One piece is wrong on purpose: code[3:] would run to the end of the string and swallow the rest.

Counting starts at zero, so position four is the fifth character. That is why the year starts at four and not at five: there is a dash at position three.

Step 2: A record is a row of fields

This is a real account line, seven fields separated by colons. Split it once, then read three of them.

Eight lines. fields is a list of seven strings, and the three reads pick out the name, the user id and the home directory. The password field sits at position one, which is why the id is at two and not at one.

This is the Python version of cut -d: -f1, with a difference worth remembering: in the shell you write a new pipeline for each field, while here you split once and index the list as often as you like.

Step 3: Join, and the way round it goes

join is the reverse of split, and it is written the way round that surprises everyone: the separator comes first.

"-".join(parts) says: take this dash and use it to join this list. parts.join("-") is the natural guess and it fails, because a list has no such method. That is worth remembering, because it is the mistake almost everyone makes first.

The separator is a string, so " ".join(parts) uses a space and "".join(parts) glues the pieces with nothing between them.

Step 4: f-strings, and a method that returns something new

Gluing strings with + gets unreadable fast and breaks when one piece is a number. An f-string puts the values inside the text.

The f before the quote is the whole trick: without it the braces are literal characters. And upper() with brackets returns a new string; line itself is untouched, which is why the first print still shows the original.

Step 5: Clean a field before you split it

Real data arrives with stray spaces. strip removes them from both ends.

Six lines. raw.strip() gives admin:user, and then the split gives the two halves. left, right = ... unpacks a two-item list into two names in one go, which is the tidy way to do it.

raw.split(":") on the uncleaned value would give " admin" with the spaces still attached, and the print would show them. That is the quiet kind of bug that only shows up in the output.

Step 6: Loop over lines and build a list

The pattern that most scripts come down to: take a list of lines, keep one field from each, collect the results.

Eight lines, and this is the shape of a hundred real scripts. names = [] starts an empty list, the loop takes each line, splits it, and appends the first field. The print at the end shows the list with its brackets and quotes, which is how Python writes a list.

names.append(line) would keep the whole line instead of the name, and the output would show the colons.

Step 7: Type one yourself

Turn a dotted address into a dashed one, using split and join together.

Step 8: A mistake to find

The code runs and prints something, but it is wrong. Press Run and look at the output closely.

It prints adminuser, with the colon gone, because the string inside the brackets is what goes between the pieces and that string is empty. Nothing raises, so the output is the only clue.

Quick reference

Next steps

• Hashing and encoding in Python is the same text handling applied to bytes.
• Reading and searching text does this job with shell tools instead.
• Text processing with sed and awk is the shell's answer to the same problem.