This question is already answered here:
- How do I make a flat list from a list? 45 answers
Is there an easy way to understand lists to flatten lists of iterative objects, otherwise, if you don't think it's the best way to flatten such shallows, balance performance and readability?
I tried to flatten such lists by understanding them as nested lists, as follows:
[image for image in menuitem for menuitem in list_of_menuitems]
But I had trouble with the NameError variant there because name'menuitem'is not defined.After Google searched for and looked at the stack overflow, I got the expected results with the reduce statement:
reduce(list.__add__, map(lambda x: list(x), list_of_menuitems))
But this method is rather unreadable because I need to call list(x) there because x is a Django QuerySet object.
Conclusion:
Thank you to everyone who contributed to this issue.This is a summary of what I have learned.If others want to add or correct these observations, I'll also make them community Wiki.
My original reduce statement is redundant, and it's best to write as follows:
>>> reduce(list.__add__, (list(mi) for mi in list_of_menuitems))
This is a nested list Deduction Correct grammar (excellent summary) dF !):
>>> [image for mi in list_of_menuitems for image in mi]
However, these methods are less efficient than using itertools.chain:
>>> from itertools import chain >>> list(chain(*list_of_menuitems))
As @cdleary points out, using chain.from_iterable may be a better style to avoid the * operator magic, as shown below:
>>> chain = itertools.chain.from_iterable([[1,2],[3],[5,89],[],[6]]) >>> print(list(chain)) >>> [1, 2, 3, 5, 89, 6]
#1st floor
stay In Python 3.4, You will be able to do the following:
[*innerlist for innerlist in outer_list]
#2nd floor
Have you tried to level it?Come from matplotlib.cbook.flatten(seq，scalarp =) ？
l=[[1,2,3],[4,5,6], [7], [8,9]]*33 run("list(flatten(l))") 3732 function calls (3303 primitive calls) in 0.007 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.007 0.007 <string>:1(<module>) 429 0.001 0.000 0.001 0.000 cbook.py:475(iterable) 429 0.002 0.000 0.003 0.000 cbook.py:484(is_string_like) 429 0.002 0.000 0.006 0.000 cbook.py:565(is_scalar_or_string) 727/298 0.001 0.000 0.007 0.000 cbook.py:605(flatten) 429 0.000 0.000 0.001 0.000 core.py:5641(isMaskedArray) 858 0.001 0.000 0.001 0.000 {isinstance} 429 0.000 0.000 0.000 0.000 {iter} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} l=[[1,2,3],[4,5,6], [7], [8,9]]*66 run("list(flatten(l))") 7461 function calls (6603 primitive calls) in 0.007 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.007 0.007 <string>:1(<module>) 858 0.001 0.000 0.001 0.000 cbook.py:475(iterable) 858 0.002 0.000 0.003 0.000 cbook.py:484(is_string_like) 858 0.002 0.000 0.006 0.000 cbook.py:565(is_scalar_or_string) 1453/595 0.001 0.000 0.007 0.000 cbook.py:605(flatten) 858 0.000 0.000 0.001 0.000 core.py:5641(isMaskedArray) 1716 0.001 0.000 0.001 0.000 {isinstance} 858 0.000 0.000 0.000 0.000 {iter} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} l=[[1,2,3],[4,5,6], [7], [8,9]]*99 run("list(flatten(l))") 11190 function calls (9903 primitive calls) in 0.010 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.010 0.010 <string>:1(<module>) 1287 0.002 0.000 0.002 0.000 cbook.py:475(iterable) 1287 0.003 0.000 0.004 0.000 cbook.py:484(is_string_like) 1287 0.002 0.000 0.009 0.000 cbook.py:565(is_scalar_or_string) 2179/892 0.001 0.000 0.010 0.000 cbook.py:605(flatten) 1287 0.001 0.000 0.001 0.000 core.py:5641(isMaskedArray) 2574 0.001 0.000 0.001 0.000 {isinstance} 1287 0.000 0.000 0.000 0.000 {iter} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} l=[[1,2,3],[4,5,6], [7], [8,9]]*132 run("list(flatten(l))") 14919 function calls (13203 primitive calls) in 0.013 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.013 0.013 <string>:1(<module>) 1716 0.002 0.000 0.002 0.000 cbook.py:475(iterable) 1716 0.004 0.000 0.006 0.000 cbook.py:484(is_string_like) 1716 0.003 0.000 0.011 0.000 cbook.py:565(is_scalar_or_string) 2905/1189 0.002 0.000 0.013 0.000 cbook.py:605(flatten) 1716 0.001 0.000 0.001 0.000 core.py:5641(isMaskedArray) 3432 0.001 0.000 0.001 0.000 {isinstance} 1716 0.001 0.000 0.001 0.000 {iter} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler'
Update gave me another idea:
l=[[1,2,3],[4,5,6], [7], [8,9]]*33 run("flattenlist(l)") 564 function calls (432 primitive calls) in 0.000 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 133/1 0.000 0.000 0.000 0.000 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.000 0.000 0.000 0.000 <string>:1(<module>) 429 0.000 0.000 0.000 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} l=[[1,2,3],[4,5,6], [7], [8,9]]*66 run("flattenlist(l)") 1125 function calls (861 primitive calls) in 0.001 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 265/1 0.001 0.000 0.001 0.001 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.000 0.000 0.001 0.001 <string>:1(<module>) 858 0.000 0.000 0.000 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} l=[[1,2,3],[4,5,6], [7], [8,9]]*99 run("flattenlist(l)") 1686 function calls (1290 primitive calls) in 0.001 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 397/1 0.001 0.000 0.001 0.001 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.000 0.000 0.001 0.001 <string>:1(<module>) 1287 0.000 0.000 0.000 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} l=[[1,2,3],[4,5,6], [7], [8,9]]*132 run("flattenlist(l)") 2247 function calls (1719 primitive calls) in 0.002 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 529/1 0.001 0.000 0.002 0.002 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.000 0.000 0.002 0.002 <string>:1(<module>) 1716 0.001 0.000 0.001 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} l=[[1,2,3],[4,5,6], [7], [8,9]]*1320 run("flattenlist(l)") 22443 function calls (17163 primitive calls) in 0.016 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 5281/1 0.011 0.000 0.016 0.016 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.000 0.000 0.016 0.016 <string>:1(<module>) 17160 0.005 0.000 0.005 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
So to test the effectiveness of recursive deepening: how much?
l=[[1,2,3],[4,5,6], [7], [8,9]]*1320 new=[l]*33 run("flattenlist(new)") 740589 function calls (566316 primitive calls) in 0.418 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 174274/1 0.281 0.000 0.417 0.417 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.001 0.001 0.418 0.418 <string>:1(<module>) 566313 0.136 0.000 0.136 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} new=[l]*66 run("flattenlist(new)") 1481175 function calls (1132629 primitive calls) in 0.809 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 348547/1 0.542 0.000 0.807 0.807 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.002 0.002 0.809 0.809 <string>:1(<module>) 1132626 0.266 0.000 0.266 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} new=[l]*99 run("flattenlist(new)") 2221761 function calls (1698942 primitive calls) in 1.211 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 522820/1 0.815 0.000 1.208 1.208 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.002 0.002 1.211 1.211 <string>:1(<module>) 1698939 0.393 0.000 0.393 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} new=[l]*132 run("flattenlist(new)") 2962347 function calls (2265255 primitive calls) in 1.630 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 697093/1 1.091 0.000 1.627 1.627 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.003 0.003 1.630 1.630 <string>:1(<module>) 2265252 0.536 0.000 0.536 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} new=[l]*1320 run("flattenlist(new)") 29623443 function calls (22652523 primitive calls) in 16.103 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 6970921/1 10.842 0.000 16.069 16.069 <ipython-input-55-39b139bad497>:4(flattenlist) 1 0.034 0.034 16.103 16.103 <string>:1(<module>) 22652520 5.227 0.000 5.227 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
I'll bet on flattenlist, and I'll use it long term instead of matploblib unless I want a revenue generator and fast results because matploblib.cbook uses flatten
That's fast.
- This is the code
:
typ=(list,tuple) def flattenlist(d): thelist = [] for x in d: if not isinstance(x,typ): thelist += [x] else: thelist += flattenlist(x) return thelist
#3rd floor
This version is a generator.Adjust the list if you want it.
def list_or_tuple(l): return isinstance(l,(list,tuple)) ## predicate will select the container to be flattened ## write your own as required ## this one flattens every list/tuple def flatten(seq,predicate=list_or_tuple): ## recursive generator for i in seq: if predicate(seq): for j in flatten(i): yield j else: yield i
You can add predicates if you want to combine predicates that meet the criteria
From python Cookbook
#4th floor
In my experience, the most effective way to flatten lists is to:
flat_list = [] map(flat_list.extend, list_of_list)
Time comparison with other proposed methods:
list_of_list = [range(10)]*1000 %timeit flat_list=[]; map(flat_list.extend, list_of_list) #10000 loops, best of 3: 119 µs per loop %timeit flat_list=list(itertools.chain.from_iterable(list_of_list)) #1000 loops, best of 3: 210 µs per loop %timeit flat_list=[i for sublist in list_of_list for i in sublist] #1000 loops, best of 3: 525 µs per loop %timeit flat_list=reduce(list.__add__,list_of_list) #100 loops, best of 3: 18.1 ms per loop
Now, when working with longer sublists, the efficiency improvements look better:
list_of_list = [range(1000)]*10 %timeit flat_list=[]; map(flat_list.extend, list_of_list) #10000 loops, best of 3: 60.7 µs per loop %timeit flat_list=list(itertools.chain.from_iterable(list_of_list)) #10000 loops, best of 3: 176 µs per loop
This method is also applicable to any iteration object:
class SquaredRange(object): def __init__(self, n): self.range = range(n) def __iter__(self): for i in self.range: yield i**2 list_of_list = [SquaredRange(5)]*3 flat_list = [] map(flat_list.extend, list_of_list) print flat_list #[0, 1, 4, 9, 16, 0, 1, 4, 9, 16, 0, 1, 4, 9, 16]
#5th floor
pylab provides a flatten: Link to numpy flatten