osm2pgsql_tuner

osm2pgsql-tuner package contains the tuner module providing the Recommendation() class.

Submodules

Package Contents

Classes

Recommendation

Takes basic inputs to generate command recommendations for osm2pgsql.

Attributes

FLAT_NODES_THRESHOLD_GB

Sets threshold size for when to use --flat-nodes.
osm2pgsql_tuner.FLAT_NODES_THRESHOLD_GB = 8.0[source]

Sets threshold size for when to use --flat-nodes.

8-10 GB appears to be an appropriate threshold when using modern SSDs for storage.

Type:

float

class osm2pgsql_tuner.Recommendation(system_ram_gb: float, osm_pbf_gb: float, slim_no_drop: bool = False, append_first_run: bool = None, pgosm_layer_set: str = 'run', ssd: bool = True)[source]

Takes basic inputs to generate command recommendations for osm2pgsql.

Parameters:

  • system_ram_gb (float) – How much total RAM the server has, in GB.

  • osm_pbf_gb (float) – Size of the .osm.pbf file in GB.

  • slim_no_drop (bool) – (Default False) Setup to use osm2pgsql --append. When true --slim must be used.

  • append_first_run (bool or None) – (Default None) If slim_no_drop is True, append_first_run must be set. Used to determine if --create or --append is passed to osm2pgsql.

  • pgosm_layer_set (str) –

    (Default (run) Base name of .lua script to run.

    PgOSM Flex uses the run entry point.

  • ssd (bool) – (Default True) Is the osm2pgsql server using SSD for storage? Value determines threshold for decision to use --flat-nodes

limited_ram_check() → bool[source]

Decide if osm2pgsql can use more RAM than the system has available.

Returns:

limited_ram

Return type:

boolean

use_flat_nodes() → bool[source]

Returns True if --flat-nodes should be used.

Use --flat-nodes when:

  • PBF size is larger than config’d threshold AND SSD

  • PBF >= 30 GB (regardless of SSD)

If the load can run entirely in-memory, no need to use flat nodes.

Returns:

use_flat_nodes

Return type:

bool

use_drop() → bool[source]

Checks other parameters to determine if --drop should be used.

Returns:

use_drop

Return type:

bool

calculate_max_osm2pgsql_cache() → float[source]

Calculates the max RAM server has available to dedicate to osm2pgsql cache.

Using 2/3 of reported system total.

osm2pgsql_cache_max : float

get_osm2pgsql_noslim_cache() → float[source]

Calculates cache required by osm2pgsql in order to run w/out slim.

Uses basic calculation based on the size of the PBF size being imported.

Justification: https://blog.rustprooflabs.com/2021/05/osm2pgsql-reduced-ram-load-to-postgis

Returns:

required_gb – Estimated memory (in GB) osm2pgsql will use if running w/out slim mode.

Return type:

float

run_in_ram() → bool[source]

Determines if bypassing --slim is an option with the given details.

Uses details about append mode, RAM available and the size of the input PBF to make determination.

Returns:

in_ram_possible

Return type:

bool

get_osm2pgsql_command(pbf_path: str) → str[source]

Builds the recommended osm2pgsql command.

Parameters:

pbf_path (str) –

Returns:

cmd

Return type:

str

get_cache_mb() → int[source]

Returns cache size to set in MB.

osm2pgsql will only use a cache value > 0 while in slim mode.

Returns:

cache – Size in MB to set –cache

Return type:

int