我有一个简单的GeoPandas数据框:



我想将此GeoDataframe上传到PostGIS表。我已经使用PostGIS扩展程序进行了数据库设置,但似乎无法将此数据框添加为表。

我尝试了以下操作:

评论

我尝试了以下操作:engine = <>#create table meta = MetaData(engine)eld_test = Table('eld_test',meta,Column('id',Integer,primary_key = True),Column('key_comb_drvr',Text) ,Column('geometry',Geometry('Point',srid = 4326)))eld_test.create(engine)#带有字典列表的DBAPI执行条件conn = engine.connect()conn.execute(eld_test.insert(),df .to_dict('records'))

欢迎使用GIS SE,请阅读我们的导览!您可以编辑您的帖子以将您的代码包含在评论中吗?

#1 楼

使用Panda的to_sql方法和SQLAlchemy,您可以在Postgres中存储数据框。并且由于您存储的是Geodataframe,GeoAlchemy将为您处理geom列。
这是一个代码示例:

# Imports
from geoalchemy2 import Geometry, WKTElement
from sqlalchemy import *
import pandas as pd
import geopandas as gpd

# Creating SQLAlchemy's engine to use
engine = create_engine('postgresql://username:password@host:socket/database')


geodataframe = gpd.GeoDataFrame(pd.DataFrame.from_csv('<your dataframe source>'))
#... [do something with the geodataframe]

geodataframe['geom'] = geodataframe['geometry'].apply(lambda x: WKTElement(x.wkt, srid=<your_SRID>)

#drop the geometry column as it is now duplicative
geodataframe.drop('geometry', 1, inplace=True)

# Use 'dtype' to specify column's type
# For the geom column, we will use GeoAlchemy's type 'Geometry'
geodataframe.to_sql(table_name, engine, if_exists='append', index=False, 
                         dtype={'geom': Geometry('POINT', srid= <your_srid>)})


值得一提的是'if_exists'参数允许您处理将数据帧添加到postgres表的方式:

    if_exists = replace: If table exists, drop it, recreate it, and insert data.
    if_exists = fail: If table exists, do nothing.
    if_exists = append: If table exists, insert data. Create if does not exist.


评论


是否有通过指定与几何列中不同的SRID在此处重新投影的机会,还是必须使用当前SRID?从几何列中获取整数SRID的最佳方法是什么?

–rovyko
20-2-16在5:53



为什么使用此方法我有sqlalchemy.exc.InvalidRequestError:无法反映:引擎错误中不存在所请求的表?

– Vilq
20-2-27在16:31

请记住,这种方法假定您的所有“ geom”实际上都填充有对象-因此,如果None类型(或某些其他非geom)占据任何记录,则将引发错误。 @Hamri Said提供的好方法!

–内森·本顿(Nathan Benton)
20 Mar 29 '20 at 19:52

我认为在这里值得一提的是,即使您使用psycopg2.connection而不是sqlalchemy.engine对象,也无法使用此方法,尽管在许多其他情况下,两者实际上是可以互换的。

– wfgeo
20年7月20日在12:55

#2 楼

我也遇到了您问过的相同问题,并花了很多天(超过我想承认的时间)寻找解决方案。假设以下带有postGIS扩展名的postgreSQL表,

postgres=> \d cldmatchup.geo_points;
Table "cldmatchup.geo_points"
Column   |         Type         |                               Modifiers                                
-----------+----------------------+------------------------------------------------------------------------
gridid    | bigint               | not null default nextval('cldmatchup.geo_points_gridid_seq'::regclass)
lat       | real                 | 
lon       | real                 | 
the_point | geography(Point,4326) | 

Indexes:
"geo_points_pkey" PRIMARY KEY, btree (gridid)


这就是我终于开始工作的地方:

import geopandas as gpd
from geoalchemy2 import Geography, Geometry
from sqlalchemy import create_engine, MetaData, Table
from sqlalchemy.orm import sessionmaker
from shapely.geometry import Point
from psycopg2.extensions import adapt, register_adapter, AsIs

# From http://initd.org/psycopg/docs/advanced.html#adapting-new-types but 
# modified to accomodate postGIS point type rather than a postgreSQL 
# point type format
def adapt_point(point):
    from psycopg2.extensions import adapt, AsIs
    x = adapt(point.x).getquoted()
    y = adapt(point.y).getquoted()
    return AsIs("'POINT (%s %s)'" % (x, y))

register_adapter(Point, adapt_point)

engine = create_engine('postgresql://<yourUserName>:postgres@localhost:5432/postgres', echo=False)
Session = sessionmaker(bind=engine)
session = Session()
meta = MetaData(engine, schema='cldmatchup')

# Create reference to pre-existing "geo_points" table in schema "cldmatchup"
geoPoints = Table('geo_points', meta, autoload=True, schema='cldmatchup', autoload_with=engine)

df = gpd.GeoDataFrame({'lat':[45.15, 35., 57.], 'lon':[-35, -150, -90.]})

# Create a shapely.geometry point 
the_point = [Point(xy) for xy in zip(df.lon, df.lat)]

# Create a GeoDataFrame specifying 'the_point' as the column with the 
# geometry data
crs = {'init': 'epsg:4326'}
geo_df = gpd.GeoDataFrame(df.copy(), crs=crs, geometry=the_point)

# Rename the geometry column to match the database table's column name.
# From https://media.readthedocs.org/pdf/geopandas/latest/geopandas.pdf,
# Section 1.2.2 p 7
geo_df = geo_df.rename(columns{'geometry':'the_point'}).set_geometry('the_point')

# Write to sql table 'geo_points'
geo_df.to_sql(geoPoints.name, engine, if_exists='append', schema='cldmatchup', index=False)

session.close()


我不能说数据库连接逻辑是否最好,因为我基本上是从另一个链接复制过来的,并且很高兴能够成功地自动映射(或反映)现有表并识别出几何定义。我已经将python编写为sql空间代码仅几个月了,所以我知道有很多东西要学习。

#3 楼

截至最近,geopandas具有to_postgis方法。 Woohoo!
注意:您将需要安装psycopg2-binarysqlalchemy2geoalchemy2
 import geopandas
from sqlalchemy import create_engine

# Set up database connection engine
engine = create_engine('postgresql://user:password@host:5432/')

# Load data into GeoDataFrame, e.g. from shapefile
geodata = geopandas.read_file("shapefile.shp")

# GeoDataFrame to PostGIS
geodata.to_postgis(
    con=engine,
    name="table_name"
)
 


#4 楼

我有一个解决方案,只需要psycopg2和shape(当然还有geopandas)。遍历(Geo)DataFrame对象通常是个坏习惯,因为它很慢,但是对于小型对象或一次性任务,仍然可以完成工作。

基本上,它通过转储几何体来工作插入时将其转换为WKB格式,然后在插入时将其重新转换为GEOMETRY类型。

请注意,您必须提前用正确的列创建表。

import psycopg2 as pg2
from shapely.wkb import dumps as wkb_dumps
import geopandas as gpd


# Assuming you already have a GeoDataFrame called "gdf"...

# Copy the gdf if you want to keep the original intact
insert_gdf = gdf.copy()

# Make a new field containing the WKB dumped from the geometry column, then turn it into a regular 
insert_gdf["geom_wkb"] = insert_gdf["geometry"].apply(lambda x: wkb_dumps(x))

# Define an insert query which will read the WKB geometry and cast it to GEOMETRY type accordingly
insert_query = """
    INSERT INTO my_table (id, geom)
    VALUES (%(id)s, ST_GeomFromWKB(%(geom_wkb)s));
"""

# Build a list of execution parameters by iterating through the GeoDataFrame
# This is considered bad practice by the pandas community because it is slow.
params_list = [
    {
        "id": i,
        "geom_wkb": row["geom_wkb"]
    } for i, row in insert_gdf.iterrows()
]

# Connect to the database and make a cursor
conn = pg2.connect(host=<your host>, port=<your port>, dbname=<your dbname>, user=<your username>, password=<your password>)
cur = conn.cursor()

# Iterate through the list of execution parameters and apply them to an execution of the insert query
for params in params_list:
    cur.execute(insert_query, params)
conn.commit()