Loading...
Loading...
Found 82 Skills
Import data into the AWS data lake from S3 files, local uploads, JDBC databases (Oracle, SQL Server, PostgreSQL, MySQL, RDS, Aurora), Amazon Redshift, Snowflake, BigQuery, DynamoDB, or existing Glue catalog tables (migration). Default target is S3 Tables; standard Iceberg on a general purpose bucket is supported where S3 Tables is not adopted. Handles one-time loads, recurring pipelines, migrations. Triggers on: import data, load data, ingest, sync database, migrate table, move data to AWS, set up pipeline, ETL, pull from Snowflake, query BigQuery into S3, export DynamoDB, CTAS, convert to Iceberg. Do NOT use for setting up or troubleshooting Glue connections (use connecting-to-data-source), creating empty tables (use creating-data-lake-table), running queries (use querying-data-lake), finding tables by fuzzy name (use finding-data-lake-assets), catalog audit (use exploring-data-catalog), or SaaS platforms like Salesforce, ServiceNow, SAP, MongoDB, Kafka.
Free 9-week data engineering course covering Docker, Terraform, Kestra, BigQuery, dbt, Spark, and Kafka with hands-on projects
Create and troubleshoot AWS Glue connections to JDBC databases (Oracle, SQL Server, PostgreSQL, MySQL, RDS), Redshift, Snowflake, and BigQuery. Gathers connection hints from user, discovers existing connections and RDS/Redshift candidates, registers credentials in Secrets Manager or IAM DB auth, configures VPC, and tests. Triggers on: connect to database, set up Glue connection, register data source, connect to Snowflake/BigQuery/RDS, connection timeout, test connection, troubleshoot connection. Do NOT use for moving data (use ingesting-into-data-lake), creating tables (use creating-data-lake-table), queries (use querying-data-lake), catalog exploration (use exploring-data-catalog), or SaaS (Salesforce, ServiceNow, SAP, MongoDB, Kafka).
DigitalOcean Managed Databases for PostgreSQL, MySQL, Redis, MongoDB, Kafka, OpenSearch, and Valkey. Use when provisioning, scaling, or operating managed database clusters on DigitalOcean.
Event-driven architecture patterns including message queues, pub/sub, event sourcing, CQRS, and sagas. Use when implementing async messaging, distributed transactions, event stores, command query separation, domain events, integration events, data streaming, choreography, orchestration, or integrating with RabbitMQ, Kafka, Apache Pulsar, AWS SQS, AWS SNS, NATS, event buses, or message brokers.
Use this skill when a user wants to store, manage, or work with Goldsky secrets — the named credential objects used by pipeline sinks. This includes: creating a new secret from a connection string or credentials, listing or inspecting existing secrets, updating or rotating credentials after a password change, and deleting secrets that are no longer needed. Trigger for any query where the user mentions 'goldsky secret', wants to securely store database credentials for a pipeline, or is working with sink authentication for PostgreSQL, Neon, Supabase, ClickHouse, Kafka, S3, Elasticsearch, DynamoDB, SQS, OpenSearch, or webhooks.
Use when you need to implement acceptance tests from a Gherkin .feature file for Spring Boot applications — including finding scenarios tagged @acceptance, implementing happy path tests with TestRestTemplate, @SpringBootTest, Testcontainers with @ServiceConnection for DB/Kafka, and WireMock for external REST stubs. Requires .feature file in context. Part of the skills-for-java project
Manages MongoDB Atlas Stream Processing (ASP) workflows. Handles workspace provisioning, data source/sink connections, processor lifecycle operations, debugging diagnostics, and tier sizing. Supports Kafka, Atlas clusters, S3, HTTPS, and Lambda integrations for streaming data workloads and event processing. NOT for general MongoDB queries or Atlas cluster management. Requires MongoDB MCP Server with Atlas API credentials.
Create or evaluate an architecture decision record (ADR). Use when choosing between technologies (e.g., Kafka vs SQS), documenting a design decision with trade-offs and consequences, reviewing a system design proposal, or designing a new component from requirements and constraints.
testcontainers-python specialist. Covers all container modules (PostgreSQL, MySQL, MongoDB, Redis, Kafka, RabbitMQ, MinIO, Elasticsearch, LocalStack), GenericContainer, wait strategies, Docker Compose, networks, pytest fixtures, and CI/CD integration. USE WHEN: user mentions "testcontainers", "docker in tests", "real database in tests", "test with real postgres/redis/kafka", asks about container fixtures or Docker-based testing. DO NOT USE FOR: Spring Boot testcontainers (Java) - use `spring-boot-integration`; Mocking HTTP - use `fastapi-testing`; Pure pytest patterns - use `pytest`
Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, implementing data governance, or troubleshooting data issues.
Complete E2E (end-to-end) and integration testing skill for TypeScript/NestJS projects using Jest, real infrastructure via Docker, and GWT pattern. ALWAYS use this skill when user needs to: **SETUP** - Initialize or configure E2E testing infrastructure: - Set up E2E testing for a new project - Configure docker-compose for testing (Kafka, PostgreSQL, MongoDB, Redis) - Create jest-e2e.config.ts or E2E Jest configuration - Set up test helpers for database, Kafka, or Redis - Configure .env.e2e environment variables - Create test/e2e directory structure **WRITE** - Create or add E2E/integration tests: - Write, create, add, or generate e2e tests or integration tests - Test API endpoints, workflows, or complete features end-to-end - Test with real databases, message brokers, or external services - Test Kafka consumers/producers, event-driven workflows - Working on any file ending in .e2e-spec.ts or in test/e2e/ directory - Use GWT (Given-When-Then) pattern for tests **REVIEW** - Audit or evaluate E2E tests: - Review existing E2E tests for quality - Check test isolation and cleanup patterns - Audit GWT pattern compliance - Evaluate assertion quality and specificity - Check for anti-patterns (multiple WHEN actions, conditional assertions) **RUN** - Execute or analyze E2E test results: - Run E2E tests - Start/stop Docker infrastructure for testing - Analyze E2E test results - Verify Docker services are healthy - Interpret test output and failures **DEBUG** - Fix failing or flaky E2E tests: - Fix failing E2E tests - Debug flaky tests or test isolation issues - Troubleshoot connection errors (database, Kafka, Redis) - Fix timeout issues or async operation failures - Diagnose race conditions or state leakage - Debug Kafka message consumption issues **OPTIMIZE** - Improve E2E test performance: - Speed up slow E2E tests - Optimize Docker infrastructure startup - Replace fixed waits with smart polling - Reduce beforeEach cleanup time - Improve test parallelization where safe Keywords: e2e, end-to-end, integration test, e2e-spec.ts, test/e2e, Jest, supertest, NestJS, Kafka, Redpanda, PostgreSQL, MongoDB, Redis, docker-compose, GWT pattern, Given-When-Then, real infrastructure, test isolation, flaky test, MSW, nock, waitForMessages, fix e2e, debug e2e, run e2e, review e2e, optimize e2e, setup e2e