Superset1.3.2 some configuration tutorials

Posted by AaZmaN on Mon, 27 Dec 2021 13:00:36 +0100

Note: the official document of superset does not support Windows system. I started to use windows, and there was an error in the configuration thumbnail later.

OS: Ubuntu20.04
Superset: 1.3.2

This is the problem I encountered in the installation and the information collected through query. The solutions to some problems may not be suitable for you, but I hope they can still help those in need.

1, Installation precautions

Here are some points I summarized during installation:

pip install Pillow -i https://pypi.douban.com/simple
# If xlrd is not installed, the page data column will not appear and upload Excel
pip install xlrd
# If mysqlclient is not installed, an error will be reported when connecting to MySQL. When I changed the data source to MySQL, because of this, I queued the error for a long time
pip install mysqlclient
pip install pymysql

# It is recommended to configure config. Before formal installation Py, change the data source to MySQL, and the current default source sqlite will be discarded later
SQLALCHEMY_DATABASE_URI = 'mysql://root:123456@localhost/superset_meta?charset=utf8'

superset load_examples execution error

For superset load during installation_ Examples: urllib error. Httperror: http error 429: too many requests error:
This is what I found online:

  • 1. Search for superset / examples from github_ Data download file set examples data master zip
  • 2. Create a folder with any name. For example, I call it example
    Then put the above examples data master Zip into the folder, attention! Be sure to unzip
  • 3. After opening the terminal, switch to the folder directory just created
  • 4. Use python to turn this folder into a shared folder
    It seems to change its permissions to this

Enter Python - M HTTP in the terminal in step 3 above server

  • 5. Open another terminal, enter ipconfig. Ubuntu is ifconfig, and find the address corresponding to IPv4 Address, such as
    192.168.157.72
  • 6. You can try to enter 192.168 directly in the browser 157.72:8000, it will appear
  • 7. Find the helper in superset py
    My directory is ~ / desktop / superset_venv/lib/python3.9/site-packages/superset/examples/helper.py

  • 8. Modify the read url address to our local address just now

    BASE_URL = "http://192.168.140.72:8000/"
    
  • 9. Rerun superset load_examples.

    This is what I changed on Windows before. It should be about the same.

2, Partial configuration description

Note: This is an online search collation

# Protect forms from CSRF attacks through flask WTF
# Flask-WTF flag for CSRF
WTF_CSRF_ENABLED = True

ENABLE_PROXY_FIX = True  # When the superset is running in the load balancer(nginx or ELB), you need to tell gunicorn which X-Forwarded-* headers can be trusted Set a series of reliable IP addresses by setting -- forwarded allow IPS

DEFAULT_FEATURE_FLAGS: = {
    # allow dashboard to use sub-domains to send chart request
    # you also need ENABLE_CORS and
    # SUPERSET_WEBSERVER_DOMAINS for list of domains
    "ALLOW_DASHBOARD_DOMAIN_SHARDING": True,  # Allow domain fragmentation of dashboard
    # Experimental feature introducing a client (browser) cache
    "CLIENT_CACHE": False,
    "DISABLE_DATASET_SOURCE_EDIT": False,  # Disable dataset editing and change to read-only mode
    # TODO CWJ
    "DYNAMIC_PLUGINS": True,  # Dynamic plug-in
    # "DYNAMIC_PLUGINS": False,
    # For some security concerns, you may need to enforce CSRF protection on
    # all query request to explore_json endpoint. In Superset, we use
    # `flask-csrf <https://sjl.bitbucket.io/flask-csrf/>`_ add csrf protection
    # for all POST requests, but this protection doesn't apply to GET method.
    # When ENABLE_EXPLORE_JSON_CSRF_PROTECTION is set to true, your users cannot
    # make GET request to explore_json. explore_json accepts both GET and POST request.
    # See `PR 7935 <https://github.com/apache/superset/pull/7935>`_ for more details.
    "ENABLE_EXPLORE_JSON_CSRF_PROTECTION": False,
    "ENABLE_TEMPLATE_PROCESSING": False,
    "ENABLE_TEMPLATE_REMOVE_FILTERS": False,
    "KV_STORE": False,
    # When this feature is enabled, nested types in Presto will be
    # expanded into extra columns and/or arrays. This is experimental,
    # and doesn't work with all nested types.
    "PRESTO_EXPAND_DATA": False,
    # Exposes API endpoint to compute thumbnails
    # TODO CWJ
    "THUMBNAILS": True,  # Open thumbnail
    # "THUMBNAILS": False,
    "DASHBOARD_CACHE": False,  # Kanban query cache
    "REMOVE_SLICE_LEVEL_LABEL_COLORS": False,
    "SHARE_QUERIES_VIA_KV_STORE": False,
    "TAGGING_SYSTEM": False,  # Enable marking function
    "SQLLAB_BACKEND_PERSISTENCE": False,
    "LISTVIEWS_DEFAULT_CARD_VIEW": False,  # Card display
    # Enables the replacement React views for all the FAB views (list, edit, show) with
    # designs introduced in https://github.com/apache/superset/issues/8976
    # (SIP-34). This is a work in progress so not all features available in FAB have
    # been implemented.
    "ENABLE_REACT_CRUD_VIEWS": True,
    # When True, this flag allows display of HTML tags in Markdown components
    "DISPLAY_MARKDOWN_HTML": True,
    # When True, this escapes HTML (rather than rendering it) in Markdown components
    "ESCAPE_MARKDOWN_HTML": False,
    # TODO CWJ
    "DASHBOARD_NATIVE_FILTERS": True,  # Open filter cascade
    "DASHBOARD_CROSS_FILTERS": True,  # Turn on the cross filter
    "DASHBOARD_NATIVE_FILTERS_SET": True,  # Filter set
    # "DASHBOARD_NATIVE_FILTERS": False,
    # "DASHBOARD_CROSS_FILTERS": False,
    # "DASHBOARD_NATIVE_FILTERS_SET": False,
    "DASHBOARD_FILTERS_EXPERIMENTAL": False,
    "GLOBAL_ASYNC_QUERIES": False,
    # TODO CWJ
    "VERSIONED_EXPORT": True,   # Enable import function
    # "VERSIONED_EXPORT": False,
    # Note that: RowLevelSecurityFilter is only given by default to the Admin role
    # and the Admin Role does have the all_datasources security permission.
    # But, if users create a specific role with access to RowLevelSecurityFilter MVC
    # and a custom datasource access, the table dropdown will not be correctly filtered
    # by that custom datasource access. So we are assuming a default security config,
    # a custom security config could potentially give access to setting filters on
    # tables that users do not have access to.
    "ROW_LEVEL_SECURITY": True,  # Row level permissions
    # Enables Alerts and reports new implementation
    # TODO CWJ
    # "ALERT_REPORTS": False,
    "ALERT_REPORTS": True,  # Alerts and reports
    # Enable experimental feature to search for other dashboards
    "OMNIBAR": False,  # Turn on drop-down visibility and keyboard commands
    "DASHBOARD_RBAC": False,
    "ENABLE_EXPLORE_DRAG_AND_DROP": False,
    # Enabling ALERTS_ATTACH_REPORTS, the system sends email and slack message
    # with screenshot and link
    # Disables ALERTS_ATTACH_REPORTS, the system DOES NOT generate screenshot
    # for report with type 'alert' and sends email and slack message with only link;
    # for report with type 'report' still send with email and slack message with
    # screenshot and link
    "ALERTS_ATTACH_REPORTS": True,
    # FORCE_DATABASE_CONNECTIONS_SSL is depreciated.
    "FORCE_DATABASE_CONNECTIONS_SSL": False,
    # Enabling ENFORCE_DB_ENCRYPTION_UI forces all database connections to be
    # encrypted before being saved into superset metastore.
    "ENFORCE_DB_ENCRYPTION_UI": False,
    # Allow users to export full CSV of table viz type.
    # This could cause the server to run out of memory or compute.
    "ALLOW_FULL_CSV_EXPORT": False,
    "UX_BETA": False,
}
    
    
    
# to configure
"ALLOW_DASHBOARD_DOMAIN_SHARDING": True,  # Allow domain fragmentation of dashboard
	# Chrome allows you to open up to six connections per domain at a time. When the dashboard has more than 6 slice s, a large number of fetch requests will queue up for the next available socket. This PR attempts to allow domain fragmentation of Superset, and this feature will be enabled only through configuration (Superset does not allow cross domain requests by default)

"CLIENT_CACHE": True,  # Enable client cache
    
"DISABLE_DATASET_SOURCE_EDIT": True,  # Add read-only mode to the source tab of the dataset editor
    # Default: everyone can edit the source:

"ENABLE_EXPLORE_JSON_CSRF_PROTECTION": False,  # Explore is allowed only through the function flag_ GET method of JSON 
    # When enable_ explore_json_ CSRF_ The protection function flag is set to True, explore_ GET in the JSON endpoint will be disabled, the backend will not perform any operations and respond with 405 MethodNotAllowed, and POST is allowed

"ENABLE_TEMPLATE_PROCESSING": True,  # Open template processing
    
"DASHBOARD_CACHE": True,  # Panel cache on
    
"TAGGING_SYSTEM": True,  # When enabled, the owner can tag the chart
    # Label chart
    # Add labels to panels
    # Note that labeling can be done automatically. Users who are not owners will see non editable labels. When you click a tag, the user will be taken to the page containing the content of the tag
    
"SQLLAB_BACKEND_PERSISTENCE": False,  # Enable backend persistence
    # If sqllab_ BACKEND_ When persistence is enabled, we will send a payload to the browser containing the user's entire query history. This PR changes the behavior so that only queries associated with an existing query editor are sent.
	# When the function flag sqllab_ BACKEND_ When persistence is set to False, user data is currently saved in local storage. Recent changes add role information to the boot user object. However, after closing the flag, the locally stored data will eventually overwrite the bootstrap data, and the role will be lost, resulting in page interruption. This change removes user data from local storage and prevents it from overwriting bootstrap data.

"LISTVIEWS_DEFAULT_CARD_VIEW": True,  # Turn on card display
    
"ENABLE_REACT_CRUD_VIEWS": True,  # Enable alternate React view for all FAB views (list, edit, display)
    # use https://github. The # designs introduced in COM / Apache / superset / issues / 8976 # (sip-34) enables the replacement React view for all FAB views (list, edit, display). This is an ongoing work, so not all the functions available in FAB have been implemented
    
"DISPLAY_MARKDOWN_HTML": True,  # HTML escapes when enabled
    # Add feature flags (use default settings to match the current behavior) (a) you can escape / display HTML code, or (b) hide the output of HTML tags
    
"ESCAPE_MARKDOWN_HTML": True,  # HTML is hidden
    # ESCAPE_MARKDOWN_HTML is escaped when HTML is enabled, and when DISPLAY_MARKDOWN_HTML is hidden when HTML is closed.
    
"DASHBOARD_NATIVE_FILTERS": True,  # Open filter cascade
"DASHBOARD_CROSS_FILTERS": True,  # Turn on the cross filter
"DASHBOARD_NATIVE_FILTERS_SET": True,  # Panel filter set
    
"GLOBAL_ASYNC_QUERIES": True,  # Global asynchronous query
    
"VERSIONED_EXPORT": True,   # Versioned export
    # When using versioned export, you can specify whether to overwrite the imported elements. (for example, dashboards, charts, datasets, databases) however, this option is only applicable to the import level (for example, overwrite only dashboards but not updated charts). When updating dashboards, you usually touch a combination of datasets, charts, and dashboards (for example, new fields, fields in the chart, more space in the dashboard for other fields), so the override option at each level may be useful. At present, you must export only modified elements 3 times (so filter the chart / dataset before exporting), and then re import them again.
    
"ROW_LEVEL_SECURITY": True,  # Open row level security directory
    
"ALERT_REPORTS": True,  # Open alarm directory
    
"OMNIBAR": True,  # Turn on drop-down visibility and keyboard commands
    # You can see it by using cmd/ctrl + k
    # See here for specific modifications. Portal: https://github.com/apache/superset/pull/16168

"DASHBOARD_RBAC": False,  # RBAC control panel
    
"ENABLE_EXPLORE_DRAG_AND_DROP": False,  # Drag and drop the POC of the query panel. Range - Groupby control only, off by default    
"ALERTS_ATTACH_REPORTS": True,  # Enable alarms to send attachments   
TALISMAN_ENABLED: Talisman

Chart RIH

1.3. Implementation ideas of versions after 0( https://github.com/askstylo/superset/pull/1)

3, Partial function configuration

Superset I configured_ config. py:

from superset.typing import CacheConfig

# Source database configuration
SQLALCHEMY_DATABASE_URI = 'mysql://root:123456@localhost/superset_meta?charset=utf8'

# Sinicization
BABEL_DEFAULT_LOCALE = "zh"

# Superset specific config
ROW_LIMIT = 5000

# SUPERSET_WEBSERVER_PORT = 8088

# Flask App Builder configuration
# Your App secret key
SECRET_KEY = '\2\1thisismyscretkey\1\2\e\y\y\h'


# Flask-WTF flag for CSRF
# Protect forms from CSRF attacks through flask WTF
WTF_CSRF_ENABLED = True
# Add endpoints that need to be exempt from CSRF protection
WTF_CSRF_EXEMPT_LIST = []
# A CSRF token that expires in 1 year
WTF_CSRF_TIME_LIMIT = 60 * 60 * 24 * 365

# Set this API key to enable Mapbox visualizations
MAPBOX_API_KEY = ''

# Global asynchronous query key
GLOBAL_ASYNC_QUERIES_JWT_SECRET = "test-secret-change-me-new-key-added-later"

# to configure
FEATURE_FLAGS = {
    # Thumbnail configuration
    "THUMBNAILS": True,
    "THUMBNAILS_SQLA_LISTENERS": True,
    # Panel cache
    "DASHBOARD_CACHE": True,
    # Global asynchronous query
    "GLOBAL_ASYNC_QUERIES": True,
    # Dynamic plug-in
    "DYNAMIC_PLUGINS": True,
    # Cross filtration
    "DASHBOARD_NATIVE_FILTERS": True,  # Open filter cascade
    "DASHBOARD_CROSS_FILTERS": True,  # Turn on the cross filter
    "DASHBOARD_NATIVE_FILTERS_SET": True,  # Filter set
    # Open import / export?? function
    "VERSIONED_EXPORT": True,
    # Alerts and reports
    "ALERT_REPORTS": True,

}

# Cache configuration
CACHE_CONFIG = {
    'CACHE_TYPE': 'redis',
    'CACHE_DEFAULT_TIMEOUT': 60 * 60 * 24,
    'CACHE_KEY_PREFIX': 'superset_',
    'CACHE_REDIS_HOST': 'localhost',
    'CACHE_REDIS_PORT': 6379,
    'CACHE_REDIS_DB': 0,
    'CACHE_REDIS_URL': 'redis://localhost:6379/0'
}

DATA_CACHE_CONFIG = {
    'CACHE_TYPE': 'redis',
    'CACHE_DEFAULT_TIMEOUT': 60 * 60 * 24,  # 1 day default (in secs)
    'CACHE_KEY_PREFIX': 'superset_results',
    'CACHE_REDIS_URL': 'redis://localhost:6379/0',
}

# Async selenium thumbnail task will use the following user
# thumbnail
THUMBNAIL_SELENIUM_USER = "admin"
THUMBNAIL_CACHE_CONFIG: CacheConfig = {
    'CACHE_TYPE': 'redis',
    'CACHE_DEFAULT_TIMEOUT': 24 * 60 * 60 * 7,
    'CACHE_KEY_PREFIX': 'thumbnail_',
    'CACHE_NO_NULL_WARNING': True,
    'CACHE_REDIS_URL': 'redis://localhost:6379/0'
}

# celery configuration
class CeleryConfig(object):
    BROKER_URL = "redis://localhost:6379/0"
    CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks", "superset.tasks.thumbnails",)
    CELERY_RESULT_BACKEND = "redis://localhost:6379/0"
    CELERYD_PREFETCH_MULTIPLIER = 10
    CELERY_ACKS_LATE = True

CELERY_CONFIG = CeleryConfig

# Resolve SQL asynchronous query errors
from cachelib.redis import RedisCache
RESULTS_BACKEND = RedisCache(
    host='localhost', port=6379, key_prefix='superset_results')

# Thumbnail required
WEBDRIVER_TYPE = "chrome"
# for older versions this was  EMAIL_REPORTS_WEBDRIVER = "chrome"
WEBDRIVER_OPTION_ARGS = [
    "--force-device-scale-factor=2.0",
    "--high-dpi-support=2.0",
    "--headless",
    "--disable-gpu",
    "--disable-dev-shm-usage",
    "--no-sandbox",
    "--disable-setuid-sandbox",
    "--disable-extensions",
]

# The base URL to query for accessing the user interface
WEBDRIVER_BASEURL = "http://localhost:8001/"

Note for thumbnails:

pip install selenium
 install chromedriver

1. Sinicization

This is executed on Windows. It should be similar in Ubuntu. I directly copy the previous projects on Windows to Ubuntu.

Open lib / site packages / superset / config. In the virtual environment Py, set BABEL_DEFAULT_LOCALE = “zh”

BABEL_DEFAULT_LOCALE = "zh"

In anaconda3 \ install \ envs \ superset_ Under the env \ lib \ site packages \ superset folder:

(superset_env) D:\Anaconda3\install\envs\superset_env\Lib\site-packages\superset>pybabel compile -d translations

Restart the file and you will find that most of them have been sinicized, but they are not completely sinicized
At this time, you need to edit: D: \ anaconda3 \ install \ envs \ superset_ env\Lib\site-packages\flask_appbuilder\translations\zh\LC_ po file under messages file, in flash_ Execute the command under the appbuilder file

pybabel compile -d translations

2. View the chart without logging in

Permission to copy Gamma, rename to Public

Add another all datasource access on all_datasource_access
among

  • can explore on Superset
  • Can explore json superset export chart json
  • all database access on all_database_access can access all databases, or you can set a single database
  • all datasource access on all_datasource_access

3.dashboard cross filter

If a filter appears on the left side of the dashboard, the dashboard can cross filter and set the following three parameters to True

    # TODO CWJ
    "DASHBOARD_NATIVE_FILTERS": True,  # Open filter cascade
    "DASHBOARD_CROSS_FILTERS": True,  # Turn on the cross filter
    "DASHBOARD_NATIVE_FILTERS_SET": True,  # Filter set

be careful dashboard Cross screening is performed by DASHBOARD_CROSS_FILTERS Control. After this is set, it is also necessary to implement cross filtering chart Up,
Check up EMIT DASHBOARD CROSS FILTERS,Not all charts have this option, pie charts table,It's on the radar chart. Are there any others,
Only some can be cross screened

The cross screening effect is shown in https://www.cnblogs.com/datawalkman/p/15131350.html Graph of

4. Log time modification

Log time difference of 8 hours

superset\models\core.py modify dttm

    # TODO CWJ
    # dttm = Column(DateTime, default=datetime.utcnow)
    dttm = Column(DateTime, default=datetime.now)

5. Introduction of CSS

6.celery,flower

pip install flower

celery --app=superset.tasks.celery_app:app worker --pool=prefork -O fair -c 4

celery --app=superset.tasks.celery_app:app flower 

7. Resolution of SQL asynchronous query error

Error: Results backend needed for asynchronous queries is not configured
Because I have configured celery and Redis when setting thumbnails earlier, this error is very strange.
solve:
Start the asynchronous query of MySQL on the web first
Then add after the cell configuration

from cachelib.redis import RedisCache
RESULTS_BACKEND = RedisCache(
    host='localhost', port=6379, key_prefix='superset_results')

# If you configure a global asynchronous query, add it again as extra 32 bits
"GLOBAL_ASYNC_QUERIES": True,
# Global asynchronous query key
GLOBAL_ASYNC_QUERIES_JWT_SECRET = "test-secret-change-me-new-key-added-later"

Topics: Python Database MySQL Visualization bi