Parsers and generator expressions

List parsing

The list parsing formula, list synthesis, is also called list derivation

#Generate a list, elements 0-9, and add the square value of 1 to each element to form a new list
x = []
for i in range(10):
    x.append((i+1)**2)
print(x

# List parsing
[(i+1)**2 for i in range(10)]

grammar

• [return value for element in iteratable object if condition]

• Brackets [] are used. The for loop is inside, and the if conditional statement is optional

• Returns a new list

List parsing is a syntax sugar

• The compiler will be optimized, which will not affect the efficiency due to abbreviation, but improve the efficiency due to optimization

• Reduce programmer workload and reduce errors

• Simplified code and enhanced readability

[expr for item in iterable if cond1 if cond2]
Equivalent to
ret = []
for item in iterable:
    if cond1:
if cond2:
ret.append(expr) 
#
[expr for i in iterable1 for j in iterable2 ]
Equivalent to
ret = []
for i in iterable1:
    for j in iterable2:
ret.append(expr)

# What are the following three outputs? Why?
[(i,j)  for i in range(7) if i>4 for j in range(20,25) if j>23]
[(i,j)  for i in range(7) for j in range(20,25) if i>4 if j>23]
[(i,j)  for i in range(7) for j in range(20,25) if i>4 and j>23]

Set analytic formula

grammar

• {return value for element in iteratable object if condition}

• When the brackets of the list parsing expression are replaced with braces {}, it becomes the set parsing expression

• Returns a collection immediately

{(x, x+1) for x in range(10)}
{[x] for x in range(10)} # OK?

Dictionary parsing

grammar

• {key:value for element in iteratable object if condition}

• The brackets of the list parsing formula are replaced by braces {}, and the element is constructed in the form of key:value

• Returns a dictionary immediately

{x:(x,x+1) for x in range(10)}
{x:[x,x+1] for x in range(10)}
{(x,):[x,x+1] for x in range(10)}
{[x]:[x,x+1] for x in range(10)} 
# {str(x):y for x in range(3) for y in range(4)} # How many elements are output?

Generator Expressions

grammar

• (return value for element in iteratable object if condition)

• Just replace the brackets of the list parsing formula with parentheses

• Returns a generator object

And list parsing

• Generator expressions are evaluated on demand (or lazy evaluation, deferred evaluation) and are evaluated only when needed

• The list parse is an immediate return value

Generator object

• Iteratable object

• iterator
  

Generator Expressions	List parsing
Delay calculation	Calculate immediately
Returns an iteratable object iterator that can iterate	Returns a list of iteratable objects, not iterators
You can only iterate once	Repeatable iteration

Comparison between generator expression and list parsing

• Calculation method

  • The generator expression delays the evaluation, and the list parsing expression evaluates immediately

• Memory usage

  • Just from the return value itself, the generator expression saves memory and the list parsing returns a new list

  • The generator has no data and occupies very little memory. When used, returning one data at a time will only occupy the space of one data

  • List parsing to construct a new list requires immediate memory consumption for all elements

• calculation speed

  • Looking at the calculation time alone, the generator expression takes a very short time and the list parsing takes a long time

  • But the generator itself does not return any value, only a generator object is returned

  • A new list is parsed and returned

summary

• Python 2 introduces list parsing

• Python2.4. Introduce generator expression

• Python 3 introduced collection and dictionary parsing, and migrated to 2.7

Generally speaking, analytic expressions should be used more, which are short and efficient. If an analytic expression is very complex and difficult to understand, consider disassembling it into a for loop.

Generators and iterators are different objects, but they are both iteratable objects.

If you do not need to get the elements of all iteratable objects immediately, in Python 3, an iterator with lazy evaluation is recommended.

Built in function	Function signature	explain
sorted	sorted(iterable[, key][, reverse])	Sort iteratable objects in ascending order by default

# Sorting must involve all participants in the container
print(sorted([1,2,3,4,5]))
print(sorted(range(10, 20), reverse=True))
print(sorted({'a':100, 'b':'abc'}))
print(sorted({'a':100, 'b':'abc'}.items()))
print(sorted({'a':'ABC', 'b':'abc'}.values(), key=str, reverse=True))
print(sorted({'a':2000, 'b':'201'}.values(), key=str))
print(sorted({'a':2000, 'b':'201'}.values(), key=int))

practice

• Give three integers, use the if statement to judge the size, and output them in ascending order

def sorter(x,y,z):
    if x>y: #x,y
        if x>z: #(x,(z.y))
            if y>z:
                return x,y,z
            else:
                return z,z,y
    else: # y,x
        if y>z: #(y,(x,z))
            if x>z:
                return y,x,z
            else:
                return y,z,x
#or
def sorter(x,y,z):
    return sorted((x,y,z),key=int,reverse=False)

• There is a list lst = [1,4,9,16,2,5,10,15], and a new list is generated. The new list element is required to be the sum of two adjacent items of lst

if __name__ == '__main__':
    lst = [1,4,9,16,2,5,10,15]  
    print([lst[i]+lst[i+1] for i in range(len(lst)-1)])

• Randomly generate 100 product ID S in the following format

• Sequential number, 6 digits, separator, dot, 10 random lowercase English characters

• For example, 00000 5 xcbaaduixy

import random
import string

if __name__ == '__main__':
    alphabet=string.ascii_lowercase
    for i in range(100):
        print('{:0>6}.{}'.format(i,''.join(random.choices(alphabet,k=10))))