feat: Production readiness improvements for WHOOSH council formation
Major security, observability, and configuration improvements:
## Security Hardening
- Implemented configurable CORS (no more wildcards)
- Added comprehensive auth middleware for admin endpoints
- Enhanced webhook HMAC validation
- Added input validation and rate limiting
- Security headers and CSP policies
## Configuration Management
- Made N8N webhook URL configurable (WHOOSH_N8N_BASE_URL)
- Replaced all hardcoded endpoints with environment variables
- Added feature flags for LLM vs heuristic composition
- Gitea fetch hardening with EAGER_FILTER and FULL_RESCAN options
## API Completeness
- Implemented GetCouncilComposition function
- Added GET /api/v1/councils/{id} endpoint
- Council artifacts API (POST/GET /api/v1/councils/{id}/artifacts)
- /admin/health/details endpoint with component status
- Database lookup for repository URLs (no hardcoded fallbacks)
## Observability & Performance
- Added OpenTelemetry distributed tracing with goal/pulse correlation
- Performance optimization database indexes
- Comprehensive health monitoring
- Enhanced logging and error handling
## Infrastructure
- Production-ready P2P discovery (replaces mock implementation)
- Removed unused Redis configuration
- Enhanced Docker Swarm integration
- Added migration files for performance indexes
## Code Quality
- Comprehensive input validation
- Graceful error handling and failsafe fallbacks
- Backwards compatibility maintained
- Following security best practices
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
39
vendor/github.com/golang-migrate/migrate/v4/database/postgres/README.md
generated
vendored
Normal file
39
vendor/github.com/golang-migrate/migrate/v4/database/postgres/README.md
generated
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
# postgres
|
||||
|
||||
`postgres://user:password@host:port/dbname?query` (`postgresql://` works, too)
|
||||
|
||||
| URL Query | WithInstance Config | Description |
|
||||
|------------|---------------------|-------------|
|
||||
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
|
||||
| `x-migrations-table-quoted` | `MigrationsTableQuoted` | By default, migrate quotes the migration table for SQL injection safety reasons. This option disable quoting and naively checks that you have quoted the migration table name. e.g. `"my_schema"."schema_migrations"` |
|
||||
| `x-statement-timeout` | `StatementTimeout` | Abort any statement that takes more than the specified number of milliseconds |
|
||||
| `x-multi-statement` | `MultiStatementEnabled` | Enable multi-statement execution (default: false) |
|
||||
| `x-multi-statement-max-size` | `MultiStatementMaxSize` | Maximum size of single statement in bytes (default: 10MB) |
|
||||
| `dbname` | `DatabaseName` | The name of the database to connect to |
|
||||
| `search_path` | | This variable specifies the order in which schemas are searched when an object is referenced by a simple name with no schema specified. |
|
||||
| `user` | | The user to sign in as |
|
||||
| `password` | | The user's password |
|
||||
| `host` | | The host to connect to. Values that start with / are for unix domain sockets. (default is localhost) |
|
||||
| `port` | | The port to bind to. (default is 5432) |
|
||||
| `fallback_application_name` | | An application_name to fall back to if one isn't provided. |
|
||||
| `connect_timeout` | | Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely. |
|
||||
| `sslcert` | | Cert file location. The file must contain PEM encoded data. |
|
||||
| `sslkey` | | Key file location. The file must contain PEM encoded data. |
|
||||
| `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
|
||||
| `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
|
||||
|
||||
|
||||
## Upgrading from v1
|
||||
|
||||
1. Write down the current migration version from schema_migrations
|
||||
1. `DROP TABLE schema_migrations`
|
||||
2. Wrap your existing migrations in transactions ([BEGIN/COMMIT](https://www.postgresql.org/docs/current/static/transaction-iso.html)) if you use multiple statements within one migration.
|
||||
3. Download and install the latest migrate version.
|
||||
4. Force the current migration version with `migrate force <current_version>`.
|
||||
|
||||
## Multi-statement mode
|
||||
|
||||
In PostgreSQL running multiple SQL statements in one `Exec` executes them inside a transaction. Sometimes this
|
||||
behavior is not desirable because some statements can be only run outside of transaction (e.g.
|
||||
`CREATE INDEX CONCURRENTLY`). If you want to use `CREATE INDEX CONCURRENTLY` without activating multi-statement mode
|
||||
you have to put such statements in a separate migration files.
|
||||
167
vendor/github.com/golang-migrate/migrate/v4/database/postgres/TUTORIAL.md
generated
vendored
Normal file
167
vendor/github.com/golang-migrate/migrate/v4/database/postgres/TUTORIAL.md
generated
vendored
Normal file
@@ -0,0 +1,167 @@
|
||||
# PostgreSQL tutorial for beginners
|
||||
|
||||
## Create/configure database
|
||||
|
||||
For the purpose of this tutorial let's create PostgreSQL database called `example`.
|
||||
Our user here is `postgres`, password `password`, and host is `localhost`.
|
||||
```
|
||||
psql -h localhost -U postgres -w -c "create database example;"
|
||||
```
|
||||
When using Migrate CLI we need to pass to database URL. Let's export it to a variable for convenience:
|
||||
```
|
||||
export POSTGRESQL_URL='postgres://postgres:password@localhost:5432/example?sslmode=disable'
|
||||
```
|
||||
`sslmode=disable` means that the connection with our database will not be encrypted. Enabling it is left as an exercise.
|
||||
|
||||
You can find further description of database URLs [here](README.md#database-urls).
|
||||
|
||||
## Create migrations
|
||||
Let's create table called `users`:
|
||||
```
|
||||
migrate create -ext sql -dir db/migrations -seq create_users_table
|
||||
```
|
||||
If there were no errors, we should have two files available under `db/migrations` folder:
|
||||
- 000001_create_users_table.down.sql
|
||||
- 000001_create_users_table.up.sql
|
||||
|
||||
Note the `sql` extension that we provided.
|
||||
|
||||
In the `.up.sql` file let's create the table:
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS users(
|
||||
user_id serial PRIMARY KEY,
|
||||
username VARCHAR (50) UNIQUE NOT NULL,
|
||||
password VARCHAR (50) NOT NULL,
|
||||
email VARCHAR (300) UNIQUE NOT NULL
|
||||
);
|
||||
```
|
||||
And in the `.down.sql` let's delete it:
|
||||
```sql
|
||||
DROP TABLE IF EXISTS users;
|
||||
```
|
||||
By adding `IF EXISTS/IF NOT EXISTS` we are making migrations idempotent - you can read more about idempotency in [getting started](../../GETTING_STARTED.md#create-migrations)
|
||||
|
||||
## Run migrations
|
||||
```
|
||||
migrate -database ${POSTGRESQL_URL} -path db/migrations up
|
||||
```
|
||||
Let's check if the table was created properly by running `psql example -c "\d users"`.
|
||||
The output you are supposed to see:
|
||||
```
|
||||
Table "public.users"
|
||||
Column | Type | Modifiers
|
||||
----------+------------------------+---------------------------------------------------------
|
||||
user_id | integer | not null default nextval('users_user_id_seq'::regclass)
|
||||
username | character varying(50) | not null
|
||||
password | character varying(50) | not null
|
||||
email | character varying(300) | not null
|
||||
Indexes:
|
||||
"users_pkey" PRIMARY KEY, btree (user_id)
|
||||
"users_email_key" UNIQUE CONSTRAINT, btree (email)
|
||||
"users_username_key" UNIQUE CONSTRAINT, btree (username)
|
||||
```
|
||||
Great! Now let's check if running reverse migration also works:
|
||||
```
|
||||
migrate -database ${POSTGRESQL_URL} -path db/migrations down
|
||||
```
|
||||
Make sure to check if your database changed as expected in this case as well.
|
||||
|
||||
## Database transactions
|
||||
|
||||
To show database transactions usage, let's create another set of migrations by running:
|
||||
```
|
||||
migrate create -ext sql -dir db/migrations -seq add_mood_to_users
|
||||
```
|
||||
Again, it should create for us two migrations files:
|
||||
- 000002_add_mood_to_users.down.sql
|
||||
- 000002_add_mood_to_users.up.sql
|
||||
|
||||
In Postgres, when we want our queries to be done in a transaction, we need to wrap it with `BEGIN` and `COMMIT` commands.
|
||||
In our example, we are going to add a column to our database that can only accept enumerable values or NULL.
|
||||
Migration up:
|
||||
```sql
|
||||
BEGIN;
|
||||
|
||||
CREATE TYPE enum_mood AS ENUM (
|
||||
'happy',
|
||||
'sad',
|
||||
'neutral'
|
||||
);
|
||||
ALTER TABLE users ADD COLUMN mood enum_mood;
|
||||
|
||||
COMMIT;
|
||||
```
|
||||
Migration down:
|
||||
```sql
|
||||
BEGIN;
|
||||
|
||||
ALTER TABLE users DROP COLUMN mood;
|
||||
DROP TYPE enum_mood;
|
||||
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
Now we can run our new migration and check the database:
|
||||
```
|
||||
migrate -database ${POSTGRESQL_URL} -path db/migrations up
|
||||
psql example -c "\d users"
|
||||
```
|
||||
Expected output:
|
||||
```
|
||||
Table "public.users"
|
||||
Column | Type | Modifiers
|
||||
----------+------------------------+---------------------------------------------------------
|
||||
user_id | integer | not null default nextval('users_user_id_seq'::regclass)
|
||||
username | character varying(50) | not null
|
||||
password | character varying(50) | not null
|
||||
email | character varying(300) | not null
|
||||
mood | enum_mood |
|
||||
Indexes:
|
||||
"users_pkey" PRIMARY KEY, btree (user_id)
|
||||
"users_email_key" UNIQUE CONSTRAINT, btree (email)
|
||||
"users_username_key" UNIQUE CONSTRAINT, btree (username)
|
||||
```
|
||||
|
||||
## Optional: Run migrations within your Go app
|
||||
Here is a very simple app running migrations for the above configuration:
|
||||
```go
|
||||
import (
|
||||
"log"
|
||||
|
||||
"github.com/golang-migrate/migrate/v4"
|
||||
_ "github.com/golang-migrate/migrate/v4/database/postgres"
|
||||
_ "github.com/golang-migrate/migrate/v4/source/file"
|
||||
)
|
||||
|
||||
func main() {
|
||||
m, err := migrate.New(
|
||||
"file://db/migrations",
|
||||
"postgres://postgres:postgres@localhost:5432/example?sslmode=disable")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
if err := m.Up(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
```
|
||||
You can find details [here](README.md#use-in-your-go-project)
|
||||
|
||||
## Fix issue where migrations run twice
|
||||
|
||||
When the schema and role names are the same, you might run into issues if you create this schema using migrations.
|
||||
This is caused by the fact that the [default `search_path`](https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH) is `"$user", public`.
|
||||
In the first run (with an empty database) the migrate table is created in `public`.
|
||||
When the migrations create the `$user` schema, the next run will store (a new) migrate table in this schema (due to order of schemas in `search_path`) and tries to apply all migrations again (most likely failing).
|
||||
|
||||
To solve this you need to change the default `search_path` by removing the `$user` component, so the migrate table is always stored in the (available) `public` schema.
|
||||
This can be done using the [`search_path` query parameter in the URL](https://github.com/jexia/migrate/blob/fix-postgres-version-table/database/postgres/README.md#postgres).
|
||||
|
||||
For example to force the migrations table in the public schema you can use:
|
||||
```
|
||||
export POSTGRESQL_URL='postgres://postgres:password@localhost:5432/example?sslmode=disable&search_path=public'
|
||||
```
|
||||
|
||||
Note that you need to explicitly add the schema names to the table names in your migrations when you to modify the tables of the non-public schema.
|
||||
|
||||
Alternatively you can add the non-public schema manually (before applying the migrations) if that is possible in your case and let the tool store the migrations table in this schema as well.
|
||||
493
vendor/github.com/golang-migrate/migrate/v4/database/postgres/postgres.go
generated
vendored
Normal file
493
vendor/github.com/golang-migrate/migrate/v4/database/postgres/postgres.go
generated
vendored
Normal file
@@ -0,0 +1,493 @@
|
||||
//go:build go1.9
|
||||
// +build go1.9
|
||||
|
||||
package postgres
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"io"
|
||||
nurl "net/url"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"go.uber.org/atomic"
|
||||
|
||||
"github.com/golang-migrate/migrate/v4"
|
||||
"github.com/golang-migrate/migrate/v4/database"
|
||||
"github.com/golang-migrate/migrate/v4/database/multistmt"
|
||||
"github.com/hashicorp/go-multierror"
|
||||
"github.com/lib/pq"
|
||||
)
|
||||
|
||||
func init() {
|
||||
db := Postgres{}
|
||||
database.Register("postgres", &db)
|
||||
database.Register("postgresql", &db)
|
||||
}
|
||||
|
||||
var (
|
||||
multiStmtDelimiter = []byte(";")
|
||||
|
||||
DefaultMigrationsTable = "schema_migrations"
|
||||
DefaultMultiStatementMaxSize = 10 * 1 << 20 // 10 MB
|
||||
)
|
||||
|
||||
var (
|
||||
ErrNilConfig = fmt.Errorf("no config")
|
||||
ErrNoDatabaseName = fmt.Errorf("no database name")
|
||||
ErrNoSchema = fmt.Errorf("no schema")
|
||||
ErrDatabaseDirty = fmt.Errorf("database is dirty")
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
MigrationsTable string
|
||||
MigrationsTableQuoted bool
|
||||
MultiStatementEnabled bool
|
||||
DatabaseName string
|
||||
SchemaName string
|
||||
migrationsSchemaName string
|
||||
migrationsTableName string
|
||||
StatementTimeout time.Duration
|
||||
MultiStatementMaxSize int
|
||||
}
|
||||
|
||||
type Postgres struct {
|
||||
// Locking and unlocking need to use the same connection
|
||||
conn *sql.Conn
|
||||
db *sql.DB
|
||||
isLocked atomic.Bool
|
||||
|
||||
// Open and WithInstance need to guarantee that config is never nil
|
||||
config *Config
|
||||
}
|
||||
|
||||
func WithConnection(ctx context.Context, conn *sql.Conn, config *Config) (*Postgres, error) {
|
||||
if config == nil {
|
||||
return nil, ErrNilConfig
|
||||
}
|
||||
|
||||
if err := conn.PingContext(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if config.DatabaseName == "" {
|
||||
query := `SELECT CURRENT_DATABASE()`
|
||||
var databaseName string
|
||||
if err := conn.QueryRowContext(ctx, query).Scan(&databaseName); err != nil {
|
||||
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
if len(databaseName) == 0 {
|
||||
return nil, ErrNoDatabaseName
|
||||
}
|
||||
|
||||
config.DatabaseName = databaseName
|
||||
}
|
||||
|
||||
if config.SchemaName == "" {
|
||||
query := `SELECT CURRENT_SCHEMA()`
|
||||
var schemaName sql.NullString
|
||||
if err := conn.QueryRowContext(ctx, query).Scan(&schemaName); err != nil {
|
||||
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
if !schemaName.Valid {
|
||||
return nil, ErrNoSchema
|
||||
}
|
||||
|
||||
config.SchemaName = schemaName.String
|
||||
}
|
||||
|
||||
if len(config.MigrationsTable) == 0 {
|
||||
config.MigrationsTable = DefaultMigrationsTable
|
||||
}
|
||||
|
||||
config.migrationsSchemaName = config.SchemaName
|
||||
config.migrationsTableName = config.MigrationsTable
|
||||
if config.MigrationsTableQuoted {
|
||||
re := regexp.MustCompile(`"(.*?)"`)
|
||||
result := re.FindAllStringSubmatch(config.MigrationsTable, -1)
|
||||
config.migrationsTableName = result[len(result)-1][1]
|
||||
if len(result) == 2 {
|
||||
config.migrationsSchemaName = result[0][1]
|
||||
} else if len(result) > 2 {
|
||||
return nil, fmt.Errorf("\"%s\" MigrationsTable contains too many dot characters", config.MigrationsTable)
|
||||
}
|
||||
}
|
||||
|
||||
px := &Postgres{
|
||||
conn: conn,
|
||||
config: config,
|
||||
}
|
||||
|
||||
if err := px.ensureVersionTable(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return px, nil
|
||||
}
|
||||
|
||||
func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
|
||||
ctx := context.Background()
|
||||
|
||||
if err := instance.Ping(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
conn, err := instance.Conn(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
px, err := WithConnection(ctx, conn, config)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
px.db = instance
|
||||
return px, nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Open(url string) (database.Driver, error) {
|
||||
purl, err := nurl.Parse(url)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
db, err := sql.Open("postgres", migrate.FilterCustomQuery(purl).String())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
migrationsTable := purl.Query().Get("x-migrations-table")
|
||||
migrationsTableQuoted := false
|
||||
if s := purl.Query().Get("x-migrations-table-quoted"); len(s) > 0 {
|
||||
migrationsTableQuoted, err = strconv.ParseBool(s)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Unable to parse option x-migrations-table-quoted: %w", err)
|
||||
}
|
||||
}
|
||||
if (len(migrationsTable) > 0) && (migrationsTableQuoted) && ((migrationsTable[0] != '"') || (migrationsTable[len(migrationsTable)-1] != '"')) {
|
||||
return nil, fmt.Errorf("x-migrations-table must be quoted (for instance '\"migrate\".\"schema_migrations\"') when x-migrations-table-quoted is enabled, current value is: %s", migrationsTable)
|
||||
}
|
||||
|
||||
statementTimeoutString := purl.Query().Get("x-statement-timeout")
|
||||
statementTimeout := 0
|
||||
if statementTimeoutString != "" {
|
||||
statementTimeout, err = strconv.Atoi(statementTimeoutString)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
multiStatementMaxSize := DefaultMultiStatementMaxSize
|
||||
if s := purl.Query().Get("x-multi-statement-max-size"); len(s) > 0 {
|
||||
multiStatementMaxSize, err = strconv.Atoi(s)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if multiStatementMaxSize <= 0 {
|
||||
multiStatementMaxSize = DefaultMultiStatementMaxSize
|
||||
}
|
||||
}
|
||||
|
||||
multiStatementEnabled := false
|
||||
if s := purl.Query().Get("x-multi-statement"); len(s) > 0 {
|
||||
multiStatementEnabled, err = strconv.ParseBool(s)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Unable to parse option x-multi-statement: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
px, err := WithInstance(db, &Config{
|
||||
DatabaseName: purl.Path,
|
||||
MigrationsTable: migrationsTable,
|
||||
MigrationsTableQuoted: migrationsTableQuoted,
|
||||
StatementTimeout: time.Duration(statementTimeout) * time.Millisecond,
|
||||
MultiStatementEnabled: multiStatementEnabled,
|
||||
MultiStatementMaxSize: multiStatementMaxSize,
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return px, nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Close() error {
|
||||
connErr := p.conn.Close()
|
||||
var dbErr error
|
||||
if p.db != nil {
|
||||
dbErr = p.db.Close()
|
||||
}
|
||||
|
||||
if connErr != nil || dbErr != nil {
|
||||
return fmt.Errorf("conn: %v, db: %v", connErr, dbErr)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// https://www.postgresql.org/docs/9.6/static/explicit-locking.html#ADVISORY-LOCKS
|
||||
func (p *Postgres) Lock() error {
|
||||
return database.CasRestoreOnErr(&p.isLocked, false, true, database.ErrLocked, func() error {
|
||||
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.migrationsSchemaName, p.config.migrationsTableName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// This will wait indefinitely until the lock can be acquired.
|
||||
query := `SELECT pg_advisory_lock($1)`
|
||||
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
|
||||
return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (p *Postgres) Unlock() error {
|
||||
return database.CasRestoreOnErr(&p.isLocked, true, false, database.ErrNotLocked, func() error {
|
||||
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.migrationsSchemaName, p.config.migrationsTableName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
query := `SELECT pg_advisory_unlock($1)`
|
||||
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (p *Postgres) Run(migration io.Reader) error {
|
||||
if p.config.MultiStatementEnabled {
|
||||
var err error
|
||||
if e := multistmt.Parse(migration, multiStmtDelimiter, p.config.MultiStatementMaxSize, func(m []byte) bool {
|
||||
if err = p.runStatement(m); err != nil {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}); e != nil {
|
||||
return e
|
||||
}
|
||||
return err
|
||||
}
|
||||
migr, err := io.ReadAll(migration)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return p.runStatement(migr)
|
||||
}
|
||||
|
||||
func (p *Postgres) runStatement(statement []byte) error {
|
||||
ctx := context.Background()
|
||||
if p.config.StatementTimeout != 0 {
|
||||
var cancel context.CancelFunc
|
||||
ctx, cancel = context.WithTimeout(ctx, p.config.StatementTimeout)
|
||||
defer cancel()
|
||||
}
|
||||
query := string(statement)
|
||||
if strings.TrimSpace(query) == "" {
|
||||
return nil
|
||||
}
|
||||
if _, err := p.conn.ExecContext(ctx, query); err != nil {
|
||||
if pgErr, ok := err.(*pq.Error); ok {
|
||||
var line uint
|
||||
var col uint
|
||||
var lineColOK bool
|
||||
if pgErr.Position != "" {
|
||||
if pos, err := strconv.ParseUint(pgErr.Position, 10, 64); err == nil {
|
||||
line, col, lineColOK = computeLineFromPos(query, int(pos))
|
||||
}
|
||||
}
|
||||
message := fmt.Sprintf("migration failed: %s", pgErr.Message)
|
||||
if lineColOK {
|
||||
message = fmt.Sprintf("%s (column %d)", message, col)
|
||||
}
|
||||
if pgErr.Detail != "" {
|
||||
message = fmt.Sprintf("%s, %s", message, pgErr.Detail)
|
||||
}
|
||||
return database.Error{OrigErr: err, Err: message, Query: statement, Line: line}
|
||||
}
|
||||
return database.Error{OrigErr: err, Err: "migration failed", Query: statement}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
|
||||
// replace crlf with lf
|
||||
s = strings.Replace(s, "\r\n", "\n", -1)
|
||||
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
|
||||
runes := []rune(s)
|
||||
if pos > len(runes) {
|
||||
return 0, 0, false
|
||||
}
|
||||
sel := runes[:pos]
|
||||
line = uint(runesCount(sel, newLine) + 1)
|
||||
col = uint(pos - 1 - runesLastIndex(sel, newLine))
|
||||
return line, col, true
|
||||
}
|
||||
|
||||
const newLine = '\n'
|
||||
|
||||
func runesCount(input []rune, target rune) int {
|
||||
var count int
|
||||
for _, r := range input {
|
||||
if r == target {
|
||||
count++
|
||||
}
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
func runesLastIndex(input []rune, target rune) int {
|
||||
for i := len(input) - 1; i >= 0; i-- {
|
||||
if input[i] == target {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func (p *Postgres) SetVersion(version int, dirty bool) error {
|
||||
tx, err := p.conn.BeginTx(context.Background(), &sql.TxOptions{})
|
||||
if err != nil {
|
||||
return &database.Error{OrigErr: err, Err: "transaction start failed"}
|
||||
}
|
||||
|
||||
query := `TRUNCATE ` + pq.QuoteIdentifier(p.config.migrationsSchemaName) + `.` + pq.QuoteIdentifier(p.config.migrationsTableName)
|
||||
if _, err := tx.Exec(query); err != nil {
|
||||
if errRollback := tx.Rollback(); errRollback != nil {
|
||||
err = multierror.Append(err, errRollback)
|
||||
}
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
// Also re-write the schema version for nil dirty versions to prevent
|
||||
// empty schema version for failed down migration on the first migration
|
||||
// See: https://github.com/golang-migrate/migrate/issues/330
|
||||
if version >= 0 || (version == database.NilVersion && dirty) {
|
||||
query = `INSERT INTO ` + pq.QuoteIdentifier(p.config.migrationsSchemaName) + `.` + pq.QuoteIdentifier(p.config.migrationsTableName) + ` (version, dirty) VALUES ($1, $2)`
|
||||
if _, err := tx.Exec(query, version, dirty); err != nil {
|
||||
if errRollback := tx.Rollback(); errRollback != nil {
|
||||
err = multierror.Append(err, errRollback)
|
||||
}
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
}
|
||||
|
||||
if err := tx.Commit(); err != nil {
|
||||
return &database.Error{OrigErr: err, Err: "transaction commit failed"}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Version() (version int, dirty bool, err error) {
|
||||
query := `SELECT version, dirty FROM ` + pq.QuoteIdentifier(p.config.migrationsSchemaName) + `.` + pq.QuoteIdentifier(p.config.migrationsTableName) + ` LIMIT 1`
|
||||
err = p.conn.QueryRowContext(context.Background(), query).Scan(&version, &dirty)
|
||||
switch {
|
||||
case err == sql.ErrNoRows:
|
||||
return database.NilVersion, false, nil
|
||||
|
||||
case err != nil:
|
||||
if e, ok := err.(*pq.Error); ok {
|
||||
if e.Code.Name() == "undefined_table" {
|
||||
return database.NilVersion, false, nil
|
||||
}
|
||||
}
|
||||
return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
|
||||
default:
|
||||
return version, dirty, nil
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Postgres) Drop() (err error) {
|
||||
// select all tables in current schema
|
||||
query := `SELECT table_name FROM information_schema.tables WHERE table_schema=(SELECT current_schema()) AND table_type='BASE TABLE'`
|
||||
tables, err := p.conn.QueryContext(context.Background(), query)
|
||||
if err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
defer func() {
|
||||
if errClose := tables.Close(); errClose != nil {
|
||||
err = multierror.Append(err, errClose)
|
||||
}
|
||||
}()
|
||||
|
||||
// delete one table after another
|
||||
tableNames := make([]string, 0)
|
||||
for tables.Next() {
|
||||
var tableName string
|
||||
if err := tables.Scan(&tableName); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(tableName) > 0 {
|
||||
tableNames = append(tableNames, tableName)
|
||||
}
|
||||
}
|
||||
if err := tables.Err(); err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
if len(tableNames) > 0 {
|
||||
// delete one by one ...
|
||||
for _, t := range tableNames {
|
||||
query = `DROP TABLE IF EXISTS ` + pq.QuoteIdentifier(t) + ` CASCADE`
|
||||
if _, err := p.conn.ExecContext(context.Background(), query); err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ensureVersionTable checks if versions table exists and, if not, creates it.
|
||||
// Note that this function locks the database, which deviates from the usual
|
||||
// convention of "caller locks" in the Postgres type.
|
||||
func (p *Postgres) ensureVersionTable() (err error) {
|
||||
if err = p.Lock(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
defer func() {
|
||||
if e := p.Unlock(); e != nil {
|
||||
if err == nil {
|
||||
err = e
|
||||
} else {
|
||||
err = multierror.Append(err, e)
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
// This block checks whether the `MigrationsTable` already exists. This is useful because it allows read only postgres
|
||||
// users to also check the current version of the schema. Previously, even if `MigrationsTable` existed, the
|
||||
// `CREATE TABLE IF NOT EXISTS...` query would fail because the user does not have the CREATE permission.
|
||||
// Taken from https://github.com/mattes/migrate/blob/master/database/postgres/postgres.go#L258
|
||||
query := `SELECT COUNT(1) FROM information_schema.tables WHERE table_schema = $1 AND table_name = $2 LIMIT 1`
|
||||
row := p.conn.QueryRowContext(context.Background(), query, p.config.migrationsSchemaName, p.config.migrationsTableName)
|
||||
|
||||
var count int
|
||||
err = row.Scan(&count)
|
||||
if err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
if count == 1 {
|
||||
return nil
|
||||
}
|
||||
|
||||
query = `CREATE TABLE IF NOT EXISTS ` + pq.QuoteIdentifier(p.config.migrationsSchemaName) + `.` + pq.QuoteIdentifier(p.config.migrationsTableName) + ` (version bigint not null primary key, dirty boolean not null)`
|
||||
if _, err = p.conn.ExecContext(context.Background(), query); err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user