[TOC]

target

List of existing dictionaries

    # 
    A = [ {dict1}, {dict2} ]
    B = [ {dict3}, {dict2} ]
    C = [ {dict3},  {dict4} ]
    M = [A,B,C]
    X = []

Put M's de duplicated dictionary into list X, and get X = [{dict1}, {dict2},{dict3}, {dict4}]

difficulty

Dictionary list

At the beginning, you may think of using set() function to convert to set and automatically de duplicate. But the set is calculated and de duplicated with hash, but the dictionary type cannot be calculated with hash. Although a class or named tuple named tuple can be used to replace the dictionary, this scenario cannot change the generation source of the list.

Method of list no set operation

The set calculation method of intersection and union difference (&, | -) cannot be used between lists

thinking

# json, poor performance
data = set([json.dumps(d) for d in data])
data = [json.loads(d) for d in data]

#  This method can only be effective for ABC, and it needs to be repeated for M, Ma FA
sortedlist = []

for item in listwhichneedssorting:
    if item not in sortedlist:
        sortedlist.append(item)

# This shortens two lines
for i in M:
    X.extend(filter(lamda s: s not in X, i)) 

# Use extend() instead of append(), because we need to splice dictionary lists instead of list lists
# lamda s: s not in X, M anonymous function, judge whether the element in i is in X
# filter() filters the above anonymous functions (i.e. duplicate dictionaries) and returns the list of dictionary elements that have not been added to X
# Append to X using extend()

application

It is mainly to take out the relationship data from neo4j, separate the nodes, connect the relationship, and convert it to the applicable data of the front end

def get_nodes_relationships(graph_list=None, ret_format=None):
    """
    //Separate relationships and nodes into their respective lists
    :param graph_list:
    :param ret_format:
    :return:
    """
    node_list = []
    relationship_list = []
    for i in map(lambda x: x.get('graph', None).get('nodes'), graph_list):
        node_list.extend(filter(lambda x: x not in node_list, i))
    for m in map(lambda y: y.get('graph', None).get('relationships', None), graph_list):
        relationship_list.extend(filter(lambda x: x not in relationship_list, m))
    # i and m are lists of dictionaries. i is a single dictionary list and M is a multi dictionary list,
    # The front end requires de duplication. Here we use the functional statement to return the dictionary that does not appear in the result list, and then use extend() to append

    # If it is for d3, part of the information needs to be changed to d3 adaptation
    if ret_format == 'd3':
        def to_d3(link):
            """
            //Change the key name of the relationship for the d3 framework, and increase the number type of the node
            :param link: relationship
            :return: Return after change
            """
            # Use the push key value pair to push again to change the key name to a source recognized by the front end
            link.update(source=link.pop('startNode'))
            # Using the push key value pair, push again to implement the change key name as the front-end recognizable target
            link.update(target=link.pop('endNode'))
            value_map = {
                "meta_in": 1,
                "slave_of": 2,
                "shard_to": 3
            }
            link['value'] = value_map[link['type']]
            return link

        relationship_list = map(lambda x: to_d3(x), relationship_list)

    # If it is for ecrats, some information needs to be changed to ecrats adaptation
    if ret_format == 'echarts':
        def to_echarts(node=None, link=None):
            """
            echarts Adaptation
            :param node: Single node
            :param link: Single relationship
            :return: Changed node or relationship
            """
            if (node and link) or (node is None and link is None):
                print("fuck you")
                exit(1)
            if node:
                node['name'] = node['id']
                node['draggable'] = True
                node['category'] = node['labels'][0]
                del node['labels']
                # del node['properties']
                bom = node
            if link:
                # Use the push key value pair to push again to change the key name to a source recognized by the front end
                link.update(source=link.pop('startNode'))
                # Using the push key value pair, push again to implement the change key name as the front-end recognizable target
                link.update(target=link.pop('endNode'))
                link.update(category=link.pop('type'))
                del link['id']
                del link['properties']
                # del link['category']
                bom = link
            return bom
        node_list = map(lambda node: to_echarts(node), node_list)
        relationship_list = map(lambda relation: to_echarts(link=relation), relationship_list)

    # Why use set instead of list to convert map objects:
    #  1. remove weight
    #  2. Reduce the object size to reduce the memory consumption
    # Why use list instead of set?
    #  1.dict object cannot be hash calculated
    ret = {"nodes": list(node_list), "links": list(relationship_list)}

    return ret