Home Spark jdbc write postgresql BatchUpdateException duplicate key value violates
Reply: 0

Spark jdbc write postgresql BatchUpdateException duplicate key value violates

dnocode
1#
dnocode Published in 2018-01-12 23:17:45Z

i'm trying to write a dataset to JDBC postgresql.

dataset
.coalesce(10).write()
            .option("numPartitions", 10)
           .mode(SaveMode.Append)
           .jdbc(getAuroraConnectionString(),
                   "table_name",auroraProp);

auroraProp has user,password and driver

I got an java.sql.BatchUpdateException: was aborted: ERROR: duplicate key value violates unique constraint

The dataset doesn't contains duplicate and before to execute a spark step i do a truncate on the table.

You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.32601 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO