Make sure that pod name is included in metrics.

by the prometheus/victoria metric crawlers.

Makefile

@@ -11,7 +11,7 @@ help:  ## display this help
 
 .PHONY: run
 run: build ## run application
-	@LOG_LEVEL=warn BLOCK_SCHEMA_INTROSPECTION=false JWT_ROLE_RATE_LIMIT=false JWT_ROLE_CLAIM_PATH="Hasura.x-hasura-default-role" JWT_USER_CLAIM_PATH="Hasura.x-hasura-user-id" HOST_GRAPHQL=https://hasura8.lan/v1/graphql ./graphql-proxy
+	@LOG_LEVEL=debug PURGE_METRICS_ON_CRAWL=true BLOCK_SCHEMA_INTROSPECTION=false CACHE_TTL=10 JWT_ROLE_RATE_LIMIT=false JWT_ROLE_CLAIM_PATH="Hasura.x-hasura-default-role" JWT_USER_CLAIM_PATH="Hasura.x-hasura-user-id" HOST_GRAPHQL=https://hasura8.lan/ HEALTHCHECK_GRAPHQL_URL=https://hasura8.lan/v1/graphql ./graphql-proxy
 
 .PHONY: build
 build: ## build the binary

README.md

@@ -1,63 +1,160 @@
 ## graphql monitoring proxy
 
-Creates a passthrough proxy to a graphql endpoint(s), allowing you for analysis of the queries and responses, producing the prometheus metrics at a fraction of the cost - because as we know - $0 is a fair price.
+Creates a passthrough proxy to a graphql endpoint(s), allowing you to analyse the queries and responses, producing the Prometheus metrics at a fraction of the cost - because, as we know - $0 is a fair price.
 
-This project is in active use by [telegram-bot.app](https://telegram-bot.app), and was tested with 30k queries per second on a single instance, consuming 10mb of RAM and 0.1% CPU.
+This project is in active use by [telegram-bot.app](https://telegram-bot.app), and was tested with 30k queries per second on a single instance, consuming 10 MB of RAM and 0.1% CPU.
 
 ![Example of monitoring dashboard](static/monitoring-at-glance.png?raw=true)
 
-You can find the example of the kubernetes manifest in the [example deployment](static/kubernetes-deployment.yaml) file.
+- [graphql monitoring proxy](#graphql-monitoring-proxy)
+  - [Why this project exists](#why-this-project-exists)
+  - [How to deploy](#how-to-deploy)
+    - [Note on websocket support](#note-on-websocket-support)
+  - [Endpoints](#endpoints)
+  - [Features](#features)
+  - [Configuration](#configuration)
+  - [Speed](#speed)
+    - [Caching](#caching)
+  - [Security](#security)
+    - [Role-based rate limiting](#role-based-rate-limiting)
+    - [Read-only mode](#read-only-mode)
+    - [Allowing access to listed URLs](#allowing-access-to-listed-urls)
+    - [Blocking introspection](#blocking-introspection)
+  - [API endpoints](#api-endpoints)
+    - [Ban or unban the user](#ban-or-unban-the-user)
+  - [General](#general)
+    - [Metrics which matter](#metrics-which-matter)
+    - [Healthcheck](#healthcheck)
+    - [Monitoring endpoint](#monitoring-endpoint)
 
 ### Why this project exists
 
-I wanted to monitor the queries and responses of our graphql endpoint, but we didn't want to pay the price of the graphql server itself ( and I will not point fingers and certain well-known project), as monitoring and basic security features should be a common, free functionality.
+I wanted to monitor the queries and responses of our graphql endpoint. Still, we didn't want to pay the price of the graphql server itself ( and I will not point fingers at a particular well-known project), as monitoring and basic security features should be a standard, free functionality.
+
+### How to deploy
+
+You can find the example of the Kubernetes manifest in the [example standalone deployment](static/kubernetes-deployment.yaml) or [example combined deployment](static/kubernetes-single-deployment.yaml) files. Observed advantage of multideployment is that it allows the network requests to travel via localhost, without leaving the deployment which brings quite significant network performance boost.
+
+#### Note on websocket support
+
+Proxy in its current version 0.5.30 does not support websockets. If you need to proxy the websocket requests - you can use following trick whilst setting up the proxy. As I'm a big fan of Traefik - there's an example which works with the mentioned above combined deployment.
+
+<details>
+  <summary>Click to show working Traefik Ingress Route example.</summary>
+
+```yaml
+apiVersion: traefik.containo.us/v1alpha1
+kind: IngressRoute
+metadata:
+  name: hasura-internal
+spec:
+  entryPoints:
+    - websecure
+  routes:
+    # NON WEBSOCKET CONNECTION
+    - kind: Rule
+      match: Host(`example.com`) && PathPrefix(`/v1/graphql`) && !HeadersRegexp(`Upgrade`, `websocket`)
+      services:
+      - name: hasura-w-proxy-internal
+        port: proxy
+      middlewares:
+        - name: compression
+          namespace: default
+
+    # WEBSOCKET CONNECTION
+    - kind: Rule
+      match: Host(`example.com`) && PathPrefix(`/v1/graphql`) && HeadersRegexp(`Upgrade`, `websocket`)
+      services:
+      - name: hasura-w-proxy-internal
+        port: hasura
+      middlewares:
+        - name: compression
+          namespace: default
+```
+
+In this case, both proxy and websockets will be available under the `/v1/graphql` path, and the websocket connection will be proxied directly to the hasura service, bypassing the proxy.
+
+</details>
 
 ### Endpoints
 
-* `:8080/v1/graphql` - the graphql endpoint
+* `:8080/*` - the graphql passthrough endpoint
 * `:9393/metrics` - the prometheus metrics endpoint
 * `:8080/healthz` - the healthcheck endpoint
+* `:8080/livez` - the liveness probe endpoint
+* `:9090/api/*` - the monitoring proxy API endpoint
 
 ### Features
 
-* MONITORING: Prometheus / VictoriaMetrics metrics
-* MONITORING: Extracting user id from JWT token and adding it as a label to the metrics
-* MONITORING: Extracting the query name and type and adding it as a label to the metrics
-* MONITORING: Calculating the query duration and adding it to the metrics
-* SPEED: Caching the queries
-* SECURITY: Blocking schema introspection
-* SECURITY: Rate limiting queries based on user role
+| Category   | Detail                                                                |
+|------------|-----------------------------------------------------------------------|
+| monitor    | Prometheus / VictoriaMetrics metrics                                  |
+| monitor    | Extracting user id from JWT token and adding it as a label to metrics |
+| monitor    | Extracting the query name and type and adding it as a label to metrics|
+| monitor    | Calculating the query duration and adding it to the metrics           |
+| speed      | Caching the queries, together with per-query cache and TTL            |
+| security   | Blocking schema introspection                                         |
+| security   | Rate limiting queries based on user role                              |
+| security   | Blocking mutations in read-only mode                                  |
+| security   | Allow access only to listed URLs                                      |
+| security   | Ban / unban specific user from accessing the application              |
+
 
 ### Configuration
 
-* `MONITORING_PORT` - the port to expose the metrics endpoint on (default: 9393)
-* `PORT_GRAPHQL` - the port to expose the graphql endpoint on (default: 8080)
-* `HOST_GRAPHQL` - the host to proxy the graphql endpoint to (default: `http://localhost/v1/graphql`)
-* `JWT_USER_CLAIM_PATH` - the path to the user claim in the JWT token (default: ``)
-* `JWT_ROLE_CLAIM_PATH` - the path to the role claim in the JWT token (default: ``)
-* `JWT_ROLE_RATE_LIMITING` - enable request rate limiting based on the role (default: `false`)
-* `ENABLE_GLOBAL_CACHE` - enable the cache (default: `false`)
-* `CACHE_TTL` - the cache TTL (default: `60s`)
-* `LOG_LEVEL` - the log level (default: `info`)
-* `BLOCK_SCHEMA_INTROSPECTION` - blocks the schema introspection (default: `false`)
-* `ENABLE_ACCESS_LOG` - enable the access log (default: `false`)
+| Parameter                 | Description                              | Default Value              |
+|---------------------------|------------------------------------------|----------------------------|
+| `MONITORING_PORT`         | The port to expose the metrics endpoint  | `9393`                     |
+| `PORT_GRAPHQL`            | The port to expose the graphql endpoint  | `8080`                     |
+| `HOST_GRAPHQL`            | The host to proxy the graphql endpoint   | `http://localhost/` |
+| `HEALTHCHECK_GRAPHQL_URL` | The URL to check the health of the graphql endpoint | `` |
+| `JWT_USER_CLAIM_PATH`     | Path to the user claim in the JWT token  | ``                         |
+| `JWT_ROLE_CLAIM_PATH`     | Path to the role claim in the JWT token  | ``                         |
+| `ROLE_FROM_HEADER`        | Header name to extract the role from     | ``                         |
+| `ROLE_RATE_LIMIT`         | Enable request rate limiting based on role| `false`                   |
+| `ENABLE_GLOBAL_CACHE`     | Enable the cache                        | `false`                    |
+| `CACHE_TTL`               | The cache TTL                           | `60`                       |
+| `LOG_LEVEL`               | The log level                           | `info`                     |
+| `BLOCK_SCHEMA_INTROSPECTION`| Blocks the schema introspection       | `false`                    |
+| `ALLOWED_INTROSPECTION`  | Allow only certain queries in introspection | ``                  |
+| `ENABLE_ACCESS_LOG`       | Enable the access log                   | `false`                    |
+| `READ_ONLY_MODE`          | Enable the read only mode               | `false`                    |
+| `ALLOWED_URLS`              | Allow access only to certain URLs       | `/v1/graphql,/v1/version`  |
+| `ENABLE_API`              | Enable the monitoring API               | `false`                    |
+| `API_PORT`                | The port to expose the monitoring API   | `9090`                     |
+| `BANNED_USERS_FILE`       | The path to the file with banned users  | `/go/src/app/banned_users.json`   |
+| `PROXIED_CLIENT_TIMEOUT` | The timeout for the proxied client in seconds     | `120`                      |
+| `PURGE_METRICS_ON_CRAWL` | Purge metrics on each /metrics crawl    | `false`                      |
 
-### Caching
+### Speed
 
-Cache engine is enabled in background as it does not use any additional resources.
-You can then start using the cache by setting the `ENABLE_GLOBAL_CACHE` environment variable to `true` - which will enable the cache for all queries, without introspection of the query. You can leave the global cache disabled and enable the cache for specific queries by adding the `@cache` directive to the query.
+#### Caching
 
-### Role based rate limiting
+The cache engine is enabled in the background by default, using no additional resources.
+You can then start using the cache by setting the `ENABLE_GLOBAL_CACHE` environment variable to `true` - which will enable the cache for all queries without introspection. You can leave the global cache disabled and enable the cache for specific queries by adding the `@cached` directive to the query.
 
-You are able to rate limit requests using the `JWT_ROLE_RATE_LIMITING` environment variable. If enabled, the proxy will rate limit the requests based on the role claim in the JWT token. You can then provide the json file in following format to specify the limits.
-Default interval is `second`, but you can use other values as well. If you want to disable the rate limiting for specific role, you can set the `req` to `0`.
+In the case of the `@cached` you can add additional parameters to the directive which will set the cache for specific queries to the provided time.
+For example, `query MyCachedQuery @cached(ttl: 90) ....` will set the cache for the query to 90 seconds.
+
+You can also set cache for specific query by using `X-Cache-Graphql-Query` header, which will set the cache for the query to the provided time, for example `X-Cache-Graphql-Query: 90` will set the cache for the query to 90 seconds.
+
+Since version `0.5.30` the cache is gzipped in the memory, which should optimise the memory usage quite significantly.
+
+### Security
+
+#### Role-based rate limiting
+
+You can rate limit requests using the `ROLE_RATE_LIMIT` environment variable. If enabled, the proxy will rate limit the requests based on the role claim in the JWT token. You can then provide the JSON file in the following format to specify the limits.
+The default interval is `second`, but you can use other values as well. If you want to disable the rate limiting for a specific role, you can set the `req` to `0`.
 
 Available values:
 `nano`, `micro`, `milli`, `second`, `minute`, `hour`, `day`
 
-To define path in JWT token where current user role is present use the `JWT_ROLE_CLAIM_PATH` environment variable.
+To define path in JWT token where the current user role is present, use the `JWT_ROLE_CLAIM_PATH` environment variable.
 
-*Default / sample configuration:*
+You can also set up the `ROLE_FROM_HEADER` environment variable to extract the role from the header instead of the JWT token. This is useful if you want to rate limit the requests for unauthenticated users. It's worth mentioning that `ROLE_FROM_HEADER` takes a priority over the `JWT_ROLE_CLAIM_PATH` environment variable and if its set, the proxy will not try to extract the role from the JWT token.
+
+*Default/sample configuration:*
 
 ```json
 {
@@ -82,7 +179,67 @@ If you'd like to change it - mount your configmap as `/app/ratelimit.json` file.
 Remember to include the `-` role, which is used for unauthenticated users or when claim can't be found for any reason.
 If rate limit has been reached - the proxy will return `429 Too Many Requests` error.
 
-### Monitoring endpoint
+
+#### Read-only mode
+
+You can enable the read-only mode by setting the `READ_ONLY_MODE` environment variable to `true` - which will block all the `mutation` queries.
+
+#### Allowing access to listed URLs
+
+You can allow access only to certain URLs by setting the `ALLOWED_URLS` environment variable to a comma-separated list of URLs. If enabled - other URLs will return `403 Forbidden` error and request will **not** reach the proxied service.
+
+#### Blocking introspection
+
+You can block the schema introspection by setting the `BLOCK_SCHEMA_INTROSPECTION` environment variable to `true` - which will block all the queries with introspection parts, like:
+
+`__schema`, `__type`, `__typename`, `__directive`, `__directivelocation`, `__field`, `__inputvalue`, `__enumvalue`, `__typekind`, `__fieldtype`, `__inputobjecttype`, `__enumtype`, `__uniontype`, `__scalars`, `__objects`, `__interfaces`, `__unions`, `__enums`, `__inputobjects`, `__directives`
+
+If you'd like to keep blocking of the schema introspection on but allow one or more of from the list of above for any reason, you can use the `ALLOWED_INTROSPECTION` environment variable to specify the list of allowed queries.
+
+`ALLOWED_INTROSPECTION="__typename,__type"`
+
+### API endpoints
+
+#### Ban or unban the user
+
+Your monitoring system can detect user misbehaving, for example trying to extract / scrap the data. To prevent user from doing so you can use the simple API to ban the user from accessing the application.
+
+To do so - you need to enable the api by setting env variable `ENABLE_API=true` which will expose the API on the port `API_PORT=9090`. Nedless to say - keep it secure and don't expose it outside of your cluster.
+
+ Then you can use the following endpoints:
+
+* `POST /api/user-ban` - ban the user from accessing the application
+* `POST /api/user-unban` - unban the user from accessing the application
+
+Both endpoints require the `user_id` parameter to be present in the request body and allow you to provide the reason for the ban.
+
+Example request:
+
+```bash
+curl -X POST \
+  http://localhost:9090/api/user-ban \
+  -H 'Content-Type: application/json' \
+  -d '{
+      "user_id": "1337",
+      "reason": "Scraping data"
+    }'
+```
+
+Ban details will be stored in the `banned_users.json` file, which you can mount as a file or configmap to the `/go/src/app/banned_users.json` path ( or use `BANNED_USERS_FILE` environment variable to specify the path to the file). The file operation is important if you have multiple instances of the proxy running, as it will allow you to ban the user from accessing the application on all instances.
+
+### General
+
+#### Metrics which matter
+
+You can always enable `PURGE_METRICS_ON_CRAWL` environment variable to purge the metrics on each `/metrics` crawl. This will allow you to see only the current metrics, without potential leftovers from the previous crawls. This is useful if you want to monitor the metrics in real-time and / or limit the amount of data ingested into the monitoring system. When enabled you will most likely need to update your monitoring queries.
+
+With the `PURGE_METRICS_ON_CRAWL` enabled, the `graphql_proxy_requests_failed`, `graphql_proxy_requests_skipped` and `graphql_proxy_requests_succesful` metrics will remain between resets.
+
+#### Healthcheck
+
+If you'd like the `/healthz` endpoint to perform actual check for the connectivity to the graphql endpoint - set the `HEALTHCHECK_GRAPHQL_URL` environment variable to the exact URL of the graphql endpoint. The query executed will be `query { __typename }` and if the response is not `200 OK` - the healthcheck will fail. Remember that the endpoint is a full URL which you'd like to check, so it should include the protocol, host and path - for example `http://localhost:8080/v1/graphql` and it's NOT the same as value of `HOST_GRAPHQL` environment variable which should provide only the host, without path, ending with slash.
+
+#### Monitoring endpoint
 
 Example metrics produced by the proxy:
 
@@ -99,4 +256,4 @@ graphql_proxy_executed_query{user_id="-",op_type="query",op_name="checkIfSpamAIR
 graphql_proxy_requests_failed 324
 graphql_proxy_requests_skipped 0
 graphql_proxy_requests_succesful 454823
-```
+```

api.go

@@ -0,0 +1,133 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"time"
+
+	fiber "github.com/gofiber/fiber/v2"
+	"github.com/gofrs/flock"
+	libpack_config "github.com/lukaszraczylo/graphql-monitoring-proxy/config"
+)
+
+var bannedUsersIDs map[string]string = make(map[string]string)
+
+func enableApi() {
+	if cfg.Server.EnableApi {
+		apiserver := fiber.New(fiber.Config{
+			DisableStartupMessage: true,
+			AppName:               fmt.Sprintf("GraphQL Monitoring Proxy - %s v%s", libpack_config.PKG_NAME, libpack_config.PKG_VERSION),
+		})
+
+		api := apiserver.Group("/api")
+		api.Post("/user-ban", apiBanUser)
+		api.Post("/user-unban", apiUnbanUser)
+
+		go periodicallyReloadBannedUsers()
+		err := apiserver.Listen(fmt.Sprintf(":%d", cfg.Server.ApiPort))
+		if err != nil {
+			cfg.Logger.Critical("Can't start the service", map[string]interface{}{"error": err.Error()})
+		}
+	}
+}
+
+func periodicallyReloadBannedUsers() {
+	for {
+		loadBannedUsers()
+		cfg.Logger.Debug("Banned users reloaded", map[string]interface{}{"users": bannedUsersIDs})
+		<-time.After(10 * time.Second)
+	}
+}
+
+func checkIfUserIsBanned(c *fiber.Ctx, userID string) bool {
+	_, found := bannedUsersIDs[userID]
+	cfg.Logger.Debug("Checking if user is banned", map[string]interface{}{"user_id": userID, "found": found})
+	if found {
+		cfg.Logger.Info("User is banned", map[string]interface{}{"user_id": userID})
+		c.Status(403).SendString("User is banned")
+	}
+	return found
+}
+
+type apiBanUserRequest struct {
+	UserID string `json:"user_id"`
+	Reason string `json:"reason"`
+}
+
+func apiBanUser(c *fiber.Ctx) error {
+	var req apiBanUserRequest
+	err := c.BodyParser(&req)
+	if err != nil {
+		cfg.Logger.Error("Can't parse the ban user request", map[string]interface{}{"error": err.Error()})
+		return err
+	}
+	bannedUsersIDs[req.UserID] = req.Reason
+	cfg.Logger.Info("Banned user", map[string]interface{}{"user_id": req.UserID, "reason": req.Reason})
+	storeBannedUsers()
+	c.Status(200).SendString("OK: user banned")
+	return nil
+}
+
+func apiUnbanUser(c *fiber.Ctx) error {
+	var req apiBanUserRequest
+	err := c.BodyParser(&req)
+	if err != nil {
+		cfg.Logger.Error("Can't parse the unban user request", map[string]interface{}{"error": err.Error()})
+		return err
+	}
+	delete(bannedUsersIDs, req.UserID)
+	cfg.Logger.Info("Unbanned user", map[string]interface{}{"user_id": req.UserID})
+	storeBannedUsers()
+	c.Status(200).SendString("OK: user unbanned")
+	return nil
+}
+
+func storeBannedUsers() {
+	fileLock := flock.New(fmt.Sprintf("%s.lock", cfg.Api.BannedUsersFile))
+	err := fileLock.Lock()
+	if err != nil {
+		cfg.Logger.Error("Can't lock the file", map[string]interface{}{"error": err.Error()})
+		return
+	}
+	defer fileLock.Unlock()
+	data, err := json.Marshal(bannedUsersIDs)
+	if err != nil {
+		cfg.Logger.Error("Can't marshal banned users", map[string]interface{}{"error": err.Error()})
+		return
+	}
+	err = os.WriteFile(cfg.Api.BannedUsersFile, data, 0644)
+	if err != nil {
+		cfg.Logger.Error("Can't write banned users to file", map[string]interface{}{"error": err.Error()})
+		return
+	}
+}
+
+func loadBannedUsers() {
+	if _, err := os.Stat(cfg.Api.BannedUsersFile); os.IsNotExist(err) {
+		cfg.Logger.Info("Banned users file doesn't exist - creating it", map[string]interface{}{"file": cfg.Api.BannedUsersFile})
+		_, err := os.Create(cfg.Api.BannedUsersFile)
+		if err != nil {
+			cfg.Logger.Error("Can't create the file", map[string]interface{}{"error": err.Error()})
+			return
+		}
+	}
+
+	fileLock := flock.New(fmt.Sprintf("%s.lock", cfg.Api.BannedUsersFile))
+	err := fileLock.RLock() // Use RLock for read lock
+	if err != nil {
+		cfg.Logger.Error("Can't lock the file [load]", map[string]interface{}{"error": err.Error()})
+		return
+	}
+	defer fileLock.Unlock()
+
+	data, err := os.ReadFile(cfg.Api.BannedUsersFile)
+	if err != nil {
+		cfg.Logger.Error("Can't read banned users from file", map[string]interface{}{"error": err.Error()})
+		return
+	}
+	err = json.Unmarshal(data, &bannedUsersIDs)
+	if err != nil {
+		cfg.Logger.Error("Can't unmarshal banned users", map[string]interface{}{"error": err.Error()})
+		return
+	}
+}

cache.go

@@ -1,33 +1,25 @@
 package main
 
 import (
-	"fmt"
 	"time"
 
-	"github.com/akyoto/cache"
 	fiber "github.com/gofiber/fiber/v2"
 	"github.com/gookit/goutil/strutil"
+	libpack_cache "github.com/lukaszraczylo/graphql-monitoring-proxy/cache"
 )
 
 func calculateHash(c *fiber.Ctx) string {
-	return strutil.Md5(fmt.Sprintf("%s", c.Body()))
+	return strutil.Md5(c.Body())
 }
 
 func enableCache() {
-	var err error
-	cfg.Cache.CacheClient = cache.New(time.Duration(cfg.Cache.CacheTTL) * time.Second * 2)
-	if err != nil {
-		cfg.Logger.Critical("Can't create cache client", map[string]interface{}{"error": err.Error()})
-		panic(err)
-	}
+	cfg.Cache.CacheClient = libpack_cache.New(time.Duration(cfg.Cache.CacheTTL) * time.Second * 100)
 }
 
 func cacheLookup(hash string) []byte {
-	if cfg.Cache.CacheClient != nil {
-		obj, found := cfg.Cache.CacheClient.Get(hash)
-		if found {
-			return obj.([]byte)
-		}
+	obj, found := cfg.Cache.CacheClient.Get(hash)
+	if found {
+		return obj
 	}
 	return nil
 }

cache/cache.go

@@ -0,0 +1,145 @@
+package libpack_cache
+
+import (
+	"bytes"
+	"compress/gzip"
+	"io"
+	"sync"
+	"time"
+)
+
+type CacheEntry struct {
+	Value     []byte
+	ExpiresAt time.Time
+}
+
+type Cache struct {
+	sync.RWMutex
+	entries   sync.Map
+	globalTTL time.Duration
+	bytePool  sync.Pool
+}
+
+func New(globalTTL time.Duration) *Cache {
+	cache := &Cache{
+		globalTTL: globalTTL,
+	}
+
+	// Initialize the byte pool.
+	cache.bytePool.New = func() interface{} {
+		return make([]byte, 0)
+	}
+
+	// Start the cache cleanup.
+	go cache.cleanupRoutine(globalTTL)
+	return cache
+}
+
+func (c *Cache) cleanupRoutine(globalTTL time.Duration) {
+	ticker := time.NewTicker(globalTTL / 2)
+	defer ticker.Stop()
+
+	for range ticker.C {
+		c.CleanExpiredEntries()
+	}
+}
+
+func (c *Cache) Set(key string, value []byte, ttl time.Duration) {
+	c.Lock()
+	defer c.Unlock()
+	expiresAt := time.Now().Add(ttl)
+
+	compressedValue, err := c.compress(value)
+	if err != nil {
+		return
+	}
+
+	// Get a byte slice from the pool and ensure it's properly sized.
+	b := c.bytePool.Get().([]byte)
+	if cap(b) < len(compressedValue) {
+		b = make([]byte, len(compressedValue))
+	} else {
+		b = b[:len(compressedValue)]
+	}
+
+	copy(b, compressedValue)
+
+	entry := CacheEntry{
+		Value:     b,
+		ExpiresAt: expiresAt,
+	}
+	c.entries.Store(key, entry)
+}
+
+func (c *Cache) Get(key string) ([]byte, bool) {
+	c.RLock()
+	defer c.RUnlock()
+
+	entry, ok := c.entries.Load(key)
+	if !ok || entry.(CacheEntry).ExpiresAt.Before(time.Now()) {
+		return nil, false
+	}
+	compressedValue := entry.(CacheEntry).Value
+	value, err := c.decompress(compressedValue)
+	if err != nil {
+		return nil, false
+	}
+
+	return value, true
+}
+
+func (c *Cache) Delete(key string) {
+	c.Lock()
+	defer c.Unlock()
+
+	entry, ok := c.entries.Load(key)
+	if !ok {
+		return
+	}
+
+	// Return the byte slice to the pool.
+	c.bytePool.Put(entry.(CacheEntry).Value)
+
+	// Delete the entry from the cache.
+	c.entries.Delete(key)
+}
+
+func (c *Cache) CleanExpiredEntries() {
+	now := time.Now()
+	c.entries.Range(func(key, value interface{}) bool {
+		entry := value.(CacheEntry)
+		if entry.ExpiresAt.Before(now) {
+			// Return the byte slice to the pool.
+			c.bytePool.Put(entry.Value)
+
+			// Delete the entry from the cache.
+			c.entries.Delete(key)
+		}
+
+		// Return true to continue iterating over the map.
+		return true
+	})
+}
+
+func (c *Cache) compress(data []byte) ([]byte, error) {
+	var buf bytes.Buffer
+	w := gzip.NewWriter(&buf)
+	_, err := w.Write(data)
+	if err != nil {
+		return nil, err
+	}
+	err = w.Close()
+	if err != nil {
+		return nil, err
+	}
+	return buf.Bytes(), nil
+}
+
+func (c *Cache) decompress(data []byte) ([]byte, error) {
+	r, err := gzip.NewReader(bytes.NewBuffer(data))
+	if err != nil {
+		return nil, err
+	}
+	defer r.Close()
+	return io.ReadAll(r)
+}

cache/cache_test.go

@@ -0,0 +1,112 @@
+package libpack_cache
+
+import (
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/suite"
+)
+
+type CacheTestSuite struct {
+	suite.Suite
+}
+
+func (suite *CacheTestSuite) SetupTest() {
+}
+
+func TestCachingTestSuite(t *testing.T) {
+	suite.Run(t, new(CacheTestSuite))
+}
+
+func (suite *CacheTestSuite) Test_New() {
+	suite.T().Run("should return a new cache", func(t *testing.T) {
+		cache := New(2 * time.Second)
+		suite.NotNil(cache)
+	})
+}
+
+func (suite *CacheTestSuite) Test_CacheUse() {
+	cache := New(30 * time.Second)
+	tests := []struct {
+		name        string
+		cache_value string
+	}{
+		{
+			name:        "test1",
+			cache_value: "test1-123",
+		},
+		{
+			name:        "test2",
+			cache_value: "test2-123",
+		},
+	}
+	for _, tt := range tests {
+		suite.T().Run(tt.name, func(t *testing.T) {
+			cache.Set(tt.name, []byte(tt.name), 5*time.Second)
+			c, ok := cache.Get(tt.name)
+			suite.Equal(true, ok)
+			suite.Equal(tt.name, string(c))
+		})
+	}
+}
+
+func (suite *CacheTestSuite) Test_CacheDelete() {
+	cache := New(30 * time.Second)
+	tests := []struct {
+		name        string
+		cache_value string
+	}{
+		{
+			name:        "test1",
+			cache_value: "test1-123",
+		},
+		{
+			name:        "test2",
+			cache_value: "test2-123",
+		},
+	}
+	for _, tt := range tests {
+		suite.T().Run(tt.name, func(t *testing.T) {
+			cache.Set(tt.name, []byte(tt.name), 5*time.Second)
+			c, ok := cache.Get(tt.name)
+			suite.Equal(true, ok)
+			suite.Equal(tt.name, string(c))
+			cache.Delete(tt.name)
+			c, ok = cache.Get(tt.name)
+			suite.Equal(false, ok)
+			suite.Equal("", string(c))
+		})
+	}
+}
+
+func (suite *CacheTestSuite) Test_CacheExpire() {
+	cache := New(30 * time.Second)
+	tests := []struct {
+		name        string
+		cache_value string
+		ttl         time.Duration
+	}{
+		{
+			name:        "test1",
+			cache_value: "test1-123",
+			ttl:         2 * time.Second,
+		},
+		{
+			name:        "test2",
+			cache_value: "test2-123",
+			ttl:         5 * time.Second,
+		},
+	}
+	for _, tt := range tests {
+		suite.T().Run(tt.name, func(t *testing.T) {
+			cache.Set(tt.name, []byte(tt.name), tt.ttl)
+			c, ok := cache.Get(tt.name)
+			suite.Equal(true, ok)
+			suite.Equal(tt.name, string(c))
+			time.Sleep(tt.ttl)
+			c, ok = cache.Get(tt.name)
+			suite.Equal(false, ok)
+			suite.Equal("", string(c))
+		})
+	}
+}

cache_test.go

@@ -27,7 +27,7 @@ func (suite *Tests) Test_cacheLookup() {
 		{
 			name: "test_existent",
 			args: args{
-				hash: "00000000000000000000000000000000000001",
+				hash: "00000000000000000000000000000000001337",
 			},
 			want: []byte("it's fine."),
 			addCache: struct {
@@ -40,7 +40,7 @@ func (suite *Tests) Test_cacheLookup() {
 	for _, tt := range tests {
 		suite.T().Run(tt.name, func(t *testing.T) {
 			if tt.addCache.data != nil {
-				cfg.Cache.CacheClient.Set(tt.args.hash, tt.addCache.data, time.Duration(1)*time.Second)
+				cfg.Cache.CacheClient.Set(tt.args.hash, tt.addCache.data, time.Duration(90*time.Second))
 			}
 			got := cacheLookup(tt.args.hash)
 			assert.Equal(tt.want, got, "Unexpected cache lookup result")

config/config.go

@@ -0,0 +1,6 @@
+package libpack_config
+
+var (
+	PKG_NAME    string = "not-specified"
+	PKG_VERSION string = "0.0.0-dev"
+)

details.go

@@ -6,7 +6,7 @@ import (
 	"strings"
 
 	"github.com/lukaszraczylo/ask"
-	libpack_monitoring "github.com/telegram-bot-app/libpack/monitoring"
+	libpack_monitoring "github.com/lukaszraczylo/graphql-monitoring-proxy/monitoring"
 )
 
 func extractClaimsFromJWTHeader(authorization string) (usr string, role string) {

go.mod

@@ -3,49 +3,48 @@ module github.com/lukaszraczylo/graphql-monitoring-proxy
 go 1.21
 
 require (
-	github.com/akyoto/cache v1.0.6
-	github.com/gofiber/fiber/v2 v2.49.2
-	github.com/gookit/goutil v0.6.12
+	github.com/VictoriaMetrics/metrics v1.24.0
+	github.com/buger/jsonparser v1.1.1
+	github.com/gofiber/fiber/v2 v2.51.0
+	github.com/gofrs/flock v0.8.1
+	github.com/google/uuid v1.4.0
+	github.com/gookit/goutil v0.6.14
 	github.com/graphql-go/graphql v0.8.1
 	github.com/json-iterator/go v1.1.12
 	github.com/lukaszraczylo/ask v0.0.0-20230927103145-2ff1123b4415
 	github.com/lukaszraczylo/go-ratecounter v0.1.8
-	github.com/lukaszraczylo/go-simple-graphql v1.1.31
+	github.com/lukaszraczylo/go-simple-graphql v1.1.37
+	github.com/rs/zerolog v1.31.0
 	github.com/stretchr/testify v1.8.4
-	github.com/telegram-bot-app/libpack v0.0.0-20231008100411-9f7f8bf94315
+	github.com/valyala/fasthttp v1.50.0
 )
 
 require (
-	dario.cat/mergo v1.0.0 // indirect
-	github.com/VictoriaMetrics/metrics v1.24.0 // indirect
-	github.com/andybalholm/brotli v1.0.5 // indirect
+	github.com/andybalholm/brotli v1.0.6 // indirect
 	github.com/avast/retry-go/v4 v4.5.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
-	github.com/google/uuid v1.3.1 // indirect
 	github.com/gookit/color v1.5.4 // indirect
-	github.com/klauspost/compress v1.17.0 // indirect
-	github.com/kr/text v0.2.0 // indirect
-	github.com/lukaszraczylo/pandati v0.0.29 // indirect
+	github.com/klauspost/compress v1.17.2 // indirect
+	github.com/kr/pretty v0.3.1 // indirect
 	github.com/mattn/go-colorable v0.1.13 // indirect
-	github.com/mattn/go-isatty v0.0.19 // indirect
+	github.com/mattn/go-isatty v0.0.20 // indirect
 	github.com/mattn/go-runewidth v0.0.15 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/rivo/uniseg v0.4.4 // indirect
-	github.com/rs/zerolog v1.31.0 // indirect
-	github.com/telegram-bot-app/lib-logging v0.0.19 // indirect
+	github.com/rogpeppe/go-internal v1.11.0 // indirect
 	github.com/valyala/bytebufferpool v1.0.0 // indirect
-	github.com/valyala/fasthttp v1.50.0 // indirect
 	github.com/valyala/fastrand v1.1.0 // indirect
 	github.com/valyala/histogram v1.2.0 // indirect
 	github.com/valyala/tcplisten v1.0.0 // indirect
-	github.com/wI2L/jsondiff v0.4.0 // indirect
 	github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
-	golang.org/x/net v0.16.0 // indirect
-	golang.org/x/sync v0.4.0 // indirect
-	golang.org/x/sys v0.13.0 // indirect
-	golang.org/x/term v0.13.0 // indirect
-	golang.org/x/text v0.13.0 // indirect
+	golang.org/x/exp v0.0.0-20231006140011-7918f672742d // indirect
+	golang.org/x/net v0.18.0 // indirect
+	golang.org/x/sync v0.5.0 // indirect
+	golang.org/x/sys v0.14.0 // indirect
+	golang.org/x/term v0.14.0 // indirect
+	golang.org/x/text v0.14.0 // indirect
+	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )

go.sum

@@ -1,11 +1,7 @@
-dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
-dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
 github.com/VictoriaMetrics/metrics v1.24.0 h1:ILavebReOjYctAGY5QU2F9X0MYvkcrG3aEn2RKa1Zkw=
 github.com/VictoriaMetrics/metrics v1.24.0/go.mod h1:eFT25kvsTidQFHb6U0oa0rTrDRdz4xTYjpL8+UPohys=
-github.com/akyoto/cache v1.0.6 h1:5XGVVYoi2i+DZLLPuVIXtsNIJ/qaAM16XT0LaBaXd2k=
-github.com/akyoto/cache v1.0.6/go.mod h1:WfxTRqKhfgAG71Xh6E3WLpjhBtZI37O53G4h5s+3iM4=
-github.com/andybalholm/brotli v1.0.5 h1:8uQZIdzKmjc/iuPu7O2ioW48L81FgatrcpfFmiq/cCs=
-github.com/andybalholm/brotli v1.0.5/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
+github.com/andybalholm/brotli v1.0.6 h1:Yf9fFpf49Zrxb9NlQaluyE92/+X7UVHlhMNJN2sxfOI=
+github.com/andybalholm/brotli v1.0.6/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
 github.com/avast/retry-go/v4 v4.5.0 h1:QoRAZZ90cj5oni2Lsgl2GW8mNTnUCnmpx/iKpwVisHg=
 github.com/avast/retry-go/v4 v4.5.0/go.mod h1:7hLEXp0oku2Nir2xBAsg0PTphp9z71bN5Aq1fboC3+I=
 github.com/buger/jsonparser v1.1.1 h1:2PnMjfWD7wBILjqQbt530v576A/cAbQvEW9gGIpYMUs=
@@ -16,38 +12,42 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
-github.com/gofiber/fiber/v2 v2.49.2 h1:ONEN3/Vc+dUCxxDgZZwpqvhISgHqb+bu+isBiEyKEQs=
-github.com/gofiber/fiber/v2 v2.49.2/go.mod h1:gNsKnyrmfEWFpJxQAV0qvW6l70K1dZGno12oLtukcts=
+github.com/gofiber/fiber/v2 v2.51.0 h1:JNACcZy5e2tGApWB2QrRpenTWn0fq0hkFm6k0C86gKQ=
+github.com/gofiber/fiber/v2 v2.51.0/go.mod h1:xaQRZQJGqnKOQnbQw+ltvku3/h8QxvNi8o6JiJ7Ll0U=
+github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
+github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
 github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
-github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
-github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4=
+github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/gookit/color v1.5.4 h1:FZmqs7XOyGgCAxmWyPslpiok1k05wmY3SJTytgvYFs0=
 github.com/gookit/color v1.5.4/go.mod h1:pZJOeOS8DM43rXbp4AZo1n9zCU2qjpcRko0b6/QJi9w=
-github.com/gookit/goutil v0.6.12 h1:73vPUcTtVGXbhSzBOFcnSB1aJl7Jq9np3RAE50yIDZc=
-github.com/gookit/goutil v0.6.12/go.mod h1:g6krlFib8xSe3G1h02IETowOtrUGpAmetT8IevDpvpM=
+github.com/gookit/goutil v0.6.14 h1:96elyOG4BvVoDaiT7vx1vHPrVyEtFfYlPPBODR0/FGQ=
+github.com/gookit/goutil v0.6.14/go.mod h1:YyDBddefmjS+mU2PDPgCcjVzTDM5WgExiDv5ZA/b8I8=
 github.com/graphql-go/graphql v0.8.1 h1:p7/Ou/WpmulocJeEx7wjQy611rtXGQaAcXGqanuMMgc=
 github.com/graphql-go/graphql v0.8.1/go.mod h1:nKiHzRM0qopJEwCITUuIsxk9PlVlwIiiI8pnJEhordQ=
 github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
-github.com/klauspost/compress v1.17.0 h1:Rnbp4K9EjcDuVuHtd0dgA4qNuv9yKDYKK1ulpJwgrqM=
-github.com/klauspost/compress v1.17.0/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
+github.com/klauspost/compress v1.17.2 h1:RlWWUY/Dr4fL8qk9YG7DTZ7PDgME2V4csBXA8L/ixi4=
+github.com/klauspost/compress v1.17.2/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
+github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
 github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
 github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/lukaszraczylo/ask v0.0.0-20230927103145-2ff1123b4415 h1:lvI8Wlbg4PxkRcg2f10wgoaRpfN19v+YdRek3+dLtlM=
 github.com/lukaszraczylo/ask v0.0.0-20230927103145-2ff1123b4415/go.mod h1:M+UVdyqZs++xtEPrascaVmZdOMhCnxjZ2SgH+xHpR0c=
 github.com/lukaszraczylo/go-ratecounter v0.1.8 h1:ZYm6Wkn58ZAlFWRmC7PaD4oAYHWcu8/0MUDWGe3PnJQ=
 github.com/lukaszraczylo/go-ratecounter v0.1.8/go.mod h1:TqXEOCtFJStk1i0tkipprv1kiDHGon1MVUisjSTBSKM=
-github.com/lukaszraczylo/go-simple-graphql v1.1.31 h1:UA3f8M1cV+XnO8UZlAqveW0qF/2NN512eB/gRqe+BHs=
-github.com/lukaszraczylo/go-simple-graphql v1.1.31/go.mod h1:MyftQ8jTdtkYImPXJpHoxz6+E53Ydv+7q9+Jr+eT8WU=
-github.com/lukaszraczylo/pandati v0.0.29 h1:WUEWm1+hWjE5KJbIL8OctG00x2dk4XKGJSlrjhxZ55k=
-github.com/lukaszraczylo/pandati v0.0.29/go.mod h1:+DyTWKFaXd+jIfe7GW5w2S5PyTko/RXxMyOa+Vl713A=
+github.com/lukaszraczylo/go-simple-graphql v1.1.37 h1:8emnfzWTApitNQ+hMHMPeGljL/xlVf/u8oKt26bUXDs=
+github.com/lukaszraczylo/go-simple-graphql v1.1.37/go.mod h1:0d/x8hj9Uus4qiv981fjL56jw8DutJ6WAeTsNWClZ/Y=
 github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
 github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
 github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
-github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
 github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
+github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
 github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=
 github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
 github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -55,12 +55,14 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w
 github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
 github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
 github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
+github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
 github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
+github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
 github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M=
 github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA=
 github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
@@ -70,10 +72,6 @@ github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
 github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
-github.com/telegram-bot-app/lib-logging v0.0.19 h1:zbyFr2ygeBY+yuaB9moXyOGk8dIBCn0jPJQjvx7YvLE=
-github.com/telegram-bot-app/lib-logging v0.0.19/go.mod h1:n8d29fRUTdgJhC4RZ8s4lP2RHiGCCRYEj2ENEClUGc8=
-github.com/telegram-bot-app/libpack v0.0.0-20231008100411-9f7f8bf94315 h1:gf+3gFgtdh48RQNmLNdK1IcGqpuTuj6RAdHxDMd/YPY=
-github.com/telegram-bot-app/libpack v0.0.0-20231008100411-9f7f8bf94315/go.mod h1:W2kWHcfNNS0r++dJ1T2XX/C4cTSxI3MsoiMbOtyqu+I=
 github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
 github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
 github.com/valyala/fasthttp v1.50.0 h1:H7fweIlBm0rXLs2q0XbalvJ6r0CUPFWK3/bB4N13e9M=
@@ -84,25 +82,23 @@ github.com/valyala/histogram v1.2.0 h1:wyYGAZZt3CpwUiIb9AU/Zbllg1llXyrtApRS815OL
 github.com/valyala/histogram v1.2.0/go.mod h1:Hb4kBwb4UxsaNbbbh+RRz8ZR6pdodR57tzWUS3BUzXY=
 github.com/valyala/tcplisten v1.0.0 h1:rBHj/Xf+E1tRGZyWIWwJDiRY0zc1Js+CV5DqwacVSA8=
 github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
-github.com/wI2L/jsondiff v0.4.0 h1:iP56F9tK83eiLttg3YdmEENtZnwlYd3ezEpNNnfZVyM=
-github.com/wI2L/jsondiff v0.4.0/go.mod h1:nR/vyy1efuDeAtMwc3AF6nZf/2LD1ID8GTyyJ+K8YB0=
 github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
 github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
 golang.org/x/exp v0.0.0-20231006140011-7918f672742d h1:jtJma62tbqLibJ5sFQz8bKtEM8rJBtfilJ2qTU199MI=
 golang.org/x/exp v0.0.0-20231006140011-7918f672742d/go.mod h1:ldy0pHrwJyGW56pPQzzkH36rKxoZW1tw7ZJpeKx+hdo=
-golang.org/x/net v0.16.0 h1:7eBu7KsSvFDtSXUIDbh3aqlK4DPsZ1rByC8PFfBThos=
-golang.org/x/net v0.16.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
-golang.org/x/sync v0.4.0 h1:zxkM55ReGkDlKSM+Fu41A+zmbZuaPVbGMzvvdUPznYQ=
-golang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
+golang.org/x/net v0.18.0 h1:mIYleuAkSbHh0tCv7RvjL3F6ZVbLjq4+R7zbOn3Kokg=
+golang.org/x/net v0.18.0/go.mod h1:/czyP5RqHAH4odGYxBJ1qz0+CE5WZ+2j1YgoEo8F2jQ=
+golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE=
+golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
-golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek=
-golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=
-golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
-golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
+golang.org/x/sys v0.14.0 h1:Vz7Qs629MkJkGyHxUlRHizWJRG2j8fbQKjELVSNhy7Q=
+golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/term v0.14.0 h1:LGK9IlZ8T9jvdy6cTdfKUCltatMFOehAQo9SRC46UQ8=
+golang.org/x/term v0.14.0/go.mod h1:TySc+nGkYR6qt8km8wUhuFRTVSMIX3XPR58y2lC8vww=
+golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
+golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=

graphql.go

@@ -1,10 +1,13 @@
 package main
 
 import (
+	"strconv"
+	"strings"
+
 	fiber "github.com/gofiber/fiber/v2"
 	"github.com/graphql-go/graphql/language/ast"
 	"github.com/graphql-go/graphql/language/parser"
-	libpack_monitoring "github.com/telegram-bot-app/libpack/monitoring"
+	libpack_monitoring "github.com/lukaszraczylo/graphql-monitoring-proxy/monitoring"
 )
 
 var retrospection_queries = []string{
@@ -33,12 +36,13 @@ var retrospection_queries = []string{
 // Saving the introspection queries as a map O(1) operation instead of O(n) for a slice.
 var retrospectionQuerySet = make(map[string]struct{}, len(retrospection_queries))
 
-func parseGraphQLQuery(c *fiber.Ctx) (operationType, operationName string, cacheRequest bool, should_block bool) {
+func parseGraphQLQuery(c *fiber.Ctx) (operationType, operationName string, cacheRequest bool, cache_time int, should_block bool, should_ignore bool) {
+	should_ignore = true
 	m := make(map[string]interface{})
 	err := json.Unmarshal(c.Body(), &m)
 	if err != nil {
-		cfg.Logger.Error("Can't unmarshal the request", map[string]interface{}{"error": err.Error(), "body": string(c.Body())})
-		cfg.Monitoring.Increment(libpack_monitoring.MetricsFailed, nil)
+		cfg.Logger.Debug("Can't unmarshal the request", map[string]interface{}{"error": err.Error(), "body": string(c.Body())})
+		cfg.Monitoring.Increment(libpack_monitoring.MetricsSkipped, nil)
 		return
 	}
 	// get the query
@@ -56,10 +60,19 @@ func parseGraphQLQuery(c *fiber.Ctx) (operationType, operationName string, cache
 		return
 	}
 
+	should_ignore = false
 	operationName = "undefined"
 	for _, d := range p.Definitions {
 		if oper, ok := d.(*ast.OperationDefinition); ok {
 			operationType = oper.Operation
+			if strings.ToLower(operationType) == "mutation" && cfg.Server.ReadOnlyMode {
+				cfg.Logger.Warning("Mutation blocked", m)
+				cfg.Monitoring.Increment(libpack_monitoring.MetricsSkipped, nil)
+				c.Status(403).SendString("The server is in read-only mode")
+				should_block = true
+				return
+			}
+
 			if oper.Name != nil {
 				operationName = oper.Name.Value
 			} else {
@@ -68,12 +81,30 @@ func parseGraphQLQuery(c *fiber.Ctx) (operationType, operationName string, cache
 			for _, dir := range oper.Directives {
 				if dir.Name.Value == "cached" {
 					cacheRequest = true
+					for _, arg := range dir.Arguments {
+						if arg.Name.Value == "ttl" {
+							cache_time, err = strconv.Atoi(arg.Value.GetValue().(string))
+							if err != nil {
+								cfg.Logger.Error("Can't parse the ttl", map[string]interface{}{"ttl": arg.Value.GetValue().(string)})
+								cfg.Monitoring.Increment(libpack_monitoring.MetricsFailed, nil)
+								return
+							}
+						}
+					}
 				}
 			}
 			if cfg.Security.BlockIntrospection {
 				for _, s := range oper.SelectionSet.Selections {
 					for _, s2 := range s.GetSelectionSet().Selections {
-						if _, exists := retrospectionQuerySet[s2.(*ast.Field).Name.Value]; exists {
+						if _, exists := retrospectionQuerySet[strings.ToLower(s2.(*ast.Field).Name.Value)]; exists {
+							if len(cfg.Security.IntrospectionAllowed) > 0 {
+								for _, introspectionQueryAllowed := range cfg.Security.IntrospectionAllowed {
+									if strings.EqualFold(strings.ToLower(introspectionQueryAllowed), strings.ToLower(s2.(*ast.Field).Name.Value)) {
+										cfg.Logger.Debug("Introspection query allowed, passing through", m)
+										return
+									}
+								}
+							}
 							cfg.Logger.Warning("Introspection query blocked", m)
 							cfg.Monitoring.Increment(libpack_monitoring.MetricsSkipped, nil)
 							c.Status(403).SendString("Introspection queries are not allowed")

logging/logging.go

@@ -0,0 +1,90 @@
+package libpack_logging
+
+import (
+	"io"
+	"os"
+	"time"
+
+	"github.com/gookit/goutil/envutil"
+	"github.com/rs/zerolog"
+)
+
+type LogConfig struct {
+	logger zerolog.Logger
+}
+
+var baseLogger zerolog.Logger
+
+func init() {
+	zerolog.TimeFieldFormat = time.RFC3339
+	zerolog.MessageFieldName = "short_message"
+	zerolog.TimestampFieldName = "timestamp"
+	zerolog.LevelFieldName = "level"
+	zerolog.LevelFatalValue = "critical"
+	baseLogger = zerolog.New(os.Stdout).With().Timestamp().Logger()
+}
+
+func NewLogger() *LogConfig {
+	switch logLevel := envutil.Getenv("LOG_LEVEL", "info"); logLevel {
+	case "debug":
+		baseLogger = baseLogger.Level(zerolog.DebugLevel)
+	case "warn":
+		baseLogger = baseLogger.Level(zerolog.WarnLevel)
+	case "error":
+		baseLogger = baseLogger.Level(zerolog.ErrorLevel)
+	default:
+		baseLogger = baseLogger.Level(zerolog.InfoLevel)
+	}
+
+	return &LogConfig{logger: baseLogger}
+}
+
+func (lw *LogConfig) log(w io.Writer, level zerolog.Level, message string, v map[string]interface{}) {
+	e := lw.logger.With().Logger()
+	e = e.Output(w)
+	event := e.WithLevel(level).CallerSkipFrame(3)
+	for k, val := range v {
+		switch v := val.(type) {
+		case string:
+			event.Str(k, v)
+		case int:
+			event.Int(k, v)
+		case float64:
+			event.Float64(k, v)
+		default:
+			event.Interface(k, val)
+		}
+	}
+	event.Msg(message)
+}
+
+func (lw *LogConfig) Debug(message string, v ...map[string]interface{}) {
+	lw.log(os.Stdout, zerolog.DebugLevel, message, mergeMaps(v))
+}
+
+func (lw *LogConfig) Info(message string, v ...map[string]interface{}) {
+	lw.log(os.Stdout, zerolog.InfoLevel, message, mergeMaps(v))
+}
+
+func (lw *LogConfig) Warning(message string, v ...map[string]interface{}) {
+	lw.log(os.Stdout, zerolog.WarnLevel, message, mergeMaps(v))
+}
+
+func (lw *LogConfig) Error(message string, v ...map[string]interface{}) {
+	lw.log(os.Stderr, zerolog.ErrorLevel, message, mergeMaps(v))
+}
+
+func (lw *LogConfig) Critical(message string, v ...map[string]interface{}) {
+	lw.log(os.Stderr, zerolog.FatalLevel, message, mergeMaps(v))
+	os.Exit(1)
+}
+
+func mergeMaps(maps []map[string]interface{}) map[string]interface{} {
+	result := make(map[string]interface{})
+	for _, m := range maps {
+		for k, v := range m {
+			result[k] = v
+		}
+	}
+	return result
+}

logging/logging_bench_test.go

@@ -0,0 +1,31 @@
+package libpack_logging
+
+import (
+	"os"
+	"testing"
+)
+
+func BenchmarkNewLogger(b *testing.B) {
+	for i := 0; i < b.N; i++ {
+		NewLogger()
+	}
+}
+
+func BenchmarkInfoLog(b *testing.B) {
+	oldEnv := os.Getenv("LOG_LEVEL")
+	os.Setenv("LOG_LEVEL", "info")
+	oldStdout := os.Stdout
+	oldStderr := os.Stderr
+	os.Stdout, _ = os.Open(os.DevNull)
+	os.Stderr, _ = os.Open(os.DevNull)
+	defer func() {
+		os.Stdout = oldStdout
+		os.Stderr = oldStderr
+		os.Setenv("LOG_LEVEL", oldEnv)
+	}()
+
+	testsLogger := NewLogger()
+	for i := 0; i < b.N; i++ {
+		testsLogger.Info("test", map[string]interface{}{"test": "test"})
+	}
+}

logging/logging_test.go

@@ -0,0 +1,373 @@
+package libpack_logging
+
+import (
+	"errors"
+	"io"
+	"os"
+	"reflect"
+	"testing"
+
+	"github.com/buger/jsonparser"
+	"github.com/stretchr/testify/suite"
+)
+
+type LoggingTestSuite struct {
+	suite.Suite
+}
+
+var (
+	testsLogger *LogConfig
+)
+
+type stdoutCapture struct {
+	oldStdout *os.File
+	readPipe  *os.File
+}
+
+func (sc *stdoutCapture) StartCapture() {
+	sc.oldStdout = os.Stdout
+	sc.readPipe, os.Stdout, _ = os.Pipe()
+}
+
+func (sc *stdoutCapture) StopCapture() (string, error) {
+	if sc.oldStdout == nil || sc.readPipe == nil {
+		return "", errors.New("StartCapture not called before StopCapture on Stdout")
+	}
+	os.Stdout.Close()
+	os.Stdout = sc.oldStdout
+	bytes, err := io.ReadAll(sc.readPipe)
+	if err != nil {
+		return "", err
+	}
+	return string(bytes), nil
+}
+
+type stderrCapture struct {
+	oldStderr *os.File
+	readPipe  *os.File
+}
+
+func (sc *stderrCapture) StartCapture() {
+	sc.oldStderr = os.Stderr
+	sc.readPipe, os.Stderr, _ = os.Pipe()
+}
+
+func (sc *stderrCapture) StopCapture() (string, error) {
+	if sc.oldStderr == nil || sc.readPipe == nil {
+		return "", errors.New("StartCapture not called before StopCapture on Stderr")
+	}
+	os.Stderr.Close()
+	os.Stderr = sc.oldStderr
+	bytes, err := io.ReadAll(sc.readPipe)
+	if err != nil {
+		return "", err
+	}
+	return string(bytes), nil
+}
+
+func (suite *LoggingTestSuite) SetupTest() {
+}
+
+func TestLoggingTestSuite(t *testing.T) {
+	suite.Run(t, new(LoggingTestSuite))
+}
+
+func (suite *LoggingTestSuite) TestLogConfig_AllHandlers() {
+	type args struct {
+		message string
+	}
+	tests := []struct {
+		name           string
+		args           args
+		wantLevel      string
+		wantMessage    string
+		envMinLogLevel string
+		loggerType     string
+		stdOutExpect   bool
+		stdErrExpect   bool
+	}{
+		{
+			name:       "Test log: Error",
+			loggerType: "Error",
+			args: args{
+				message: "This is a error message",
+			},
+			wantLevel:    "error",
+			wantMessage:  "This is a error message",
+			stdErrExpect: true,
+			stdOutExpect: false,
+		},
+		{
+			name:       "Test log: Warning",
+			loggerType: "Warning",
+			args: args{
+				message: "This is a warning message",
+			},
+			wantLevel:      "warn",
+			wantMessage:    "This is a warning message",
+			stdErrExpect:   false,
+			stdOutExpect:   true,
+			envMinLogLevel: "info",
+		},
+		{
+			name:       "Test log: Warning | Min level: Debug",
+			loggerType: "Warning",
+			args: args{
+				message: "This is a warning message",
+			},
+			wantLevel:      "warn",
+			wantMessage:    "This is a warning message",
+			stdErrExpect:   false,
+			stdOutExpect:   true,
+			envMinLogLevel: "debug",
+		},
+		{
+			name:       "Test log: Info",
+			loggerType: "Info",
+			args: args{
+				message: "This is a info message",
+			},
+			wantLevel:    "info",
+			wantMessage:  "This is a info message",
+			stdErrExpect: false,
+			stdOutExpect: true,
+		},
+		{
+			name:       "Test log: Info | Min level: Warn",
+			loggerType: "Info",
+			args: args{
+				message: "This is a info message",
+			},
+			wantLevel:      "",
+			wantMessage:    "",
+			stdErrExpect:   false,
+			stdOutExpect:   false,
+			envMinLogLevel: "warn",
+		},
+		{
+			name:       "Test log: Warning | Min level: Warn",
+			loggerType: "Warning",
+			args: args{
+				message: "This is a warning message",
+			},
+			wantLevel:      "warn",
+			wantMessage:    "This is a warning message",
+			stdErrExpect:   false,
+			stdOutExpect:   true,
+			envMinLogLevel: "warn",
+		},
+		{
+			name:       "Test log: Warning | Min level: Error",
+			loggerType: "Warning",
+			args: args{
+				message: "This is an error message",
+			},
+			wantLevel:      "",
+			wantMessage:    "",
+			stdErrExpect:   false,
+			stdOutExpect:   false,
+			envMinLogLevel: "error",
+		},
+		{
+			name:       "Test log: Debug | Min level: Debug",
+			loggerType: "Debug",
+			args: args{
+				message: "This is a debug message",
+			},
+			wantLevel:      "debug",
+			wantMessage:    "This is a debug message",
+			stdErrExpect:   false,
+			stdOutExpect:   true,
+			envMinLogLevel: "debug",
+		},
+	}
+
+	for _, tt := range tests {
+		suite.T().Run(tt.name, func(t *testing.T) {
+			if tt.envMinLogLevel != "" {
+				os.Setenv("LOG_LEVEL", tt.envMinLogLevel)
+				defer os.Unsetenv("LOG_LEVEL")
+			}
+			testsLogger = NewLogger()
+
+			captureStdout := stdoutCapture{}
+			captureStdout.StartCapture()
+			captureStderr := stderrCapture{}
+			captureStderr.StartCapture()
+
+			reflect.ValueOf(testsLogger).MethodByName(tt.loggerType).Call([]reflect.Value{reflect.ValueOf(tt.args.message)})
+
+			stdoutOut, err := captureStdout.StopCapture()
+			if err != nil {
+				suite.T().Fatal(err)
+			}
+
+			stderrOut, err := captureStderr.StopCapture()
+			if err != nil {
+				suite.T().Fatal(err)
+			}
+
+			if tt.stdErrExpect && !tt.stdOutExpect {
+				gotLvl, gotMsg, err := getResponseValues(stderrOut, "short_message")
+				suite.NoError(err, "Failed in [STDERR]: "+tt.name)
+				suite.Equal(tt.wantLevel, gotLvl, "Failed in [STDERR]: "+tt.name)
+				suite.Equal(tt.wantMessage, gotMsg, "Failed in [STDERR]: "+tt.name)
+				suite.Equal("", stdoutOut, "Failed in [STDERR]: "+tt.name)
+			}
+			if tt.stdOutExpect && !tt.stdErrExpect {
+				gotLvl, gotMsg, err := getResponseValues(stdoutOut, "short_message")
+				suite.NoError(err, "Failed in [STDOUT]: "+tt.name)
+				suite.Equal(tt.wantLevel, gotLvl, "Failed in [STDOUT]: "+tt.name)
+				suite.Equal(tt.wantMessage, gotMsg, "Failed in [STDOUT]: "+tt.name)
+				suite.Equal("", stderrOut, "Failed in [STDOUT]: "+tt.name)
+			}
+			if !tt.stdErrExpect && !tt.stdOutExpect {
+				suite.Equal("", stderrOut, "Failed in [NEITHER]: "+tt.name)
+				suite.Equal("", stdoutOut, "Failed in [NEITHER]: "+tt.name)
+			}
+			os.Unsetenv("LOG_LEVEL")
+		})
+	}
+}
+
+func (suite *LoggingTestSuite) TestFullMessage() {
+	type args struct {
+		extraFields map[string]interface{}
+		message     string
+	}
+	extraFields := make(map[string]interface{})
+	extraFields["_full_message"] = "full message"
+
+	tests := []struct {
+		args           args
+		name           string
+		wantLevel      string
+		wantMessage    string
+		envMinLogLevel string
+		loggerType     string
+		stdOutExpect   bool
+		stdErrExpect   bool
+	}{
+		{
+			name:       "Test log: Error",
+			loggerType: "Error",
+			args: args{
+				message:     "This is a error message",
+				extraFields: extraFields,
+			},
+			wantLevel:    "error",
+			wantMessage:  extraFields["_full_message"].(string),
+			stdErrExpect: true,
+			stdOutExpect: false,
+		},
+		{
+			name:       "Test log: Info",
+			loggerType: "Info",
+			args: args{
+				message:     "This is a info message",
+				extraFields: extraFields,
+			},
+			wantMessage:  extraFields["_full_message"].(string),
+			stdErrExpect: false,
+			stdOutExpect: true,
+		},
+	}
+
+	for _, tt := range tests {
+		suite.T().Run(tt.name, func(t *testing.T) {
+			if tt.envMinLogLevel != "" {
+				os.Setenv("LOG_LEVEL", tt.envMinLogLevel)
+				defer os.Unsetenv("LOG_LEVEL")
+			}
+			testsLogger = NewLogger()
+
+			captureStdout := stdoutCapture{}
+			captureStdout.StartCapture()
+			captureStderr := stderrCapture{}
+			captureStderr.StartCapture()
+
+			reflect.ValueOf(testsLogger).MethodByName(tt.loggerType).Call([]reflect.Value{
+				reflect.ValueOf(tt.args.message),
+				reflect.ValueOf(tt.args.extraFields),
+			})
+
+			stdoutOut, err := captureStdout.StopCapture()
+			if err != nil {
+				suite.T().Fatal(err)
+			}
+
+			stderrOut, err := captureStderr.StopCapture()
+			if err != nil {
+				suite.T().Fatal(err)
+			}
+
+			if tt.stdErrExpect && !tt.stdOutExpect {
+				_, gotMsg, err := getResponseValues(stderrOut, "_full_message")
+				suite.NoError(err, "Failed in [STDERR]: "+tt.name)
+				suite.Equal(tt.wantMessage, gotMsg, "Failed in [STDERR]: "+tt.name)
+			}
+			if tt.stdOutExpect && !tt.stdErrExpect {
+				_, gotMsg, err := getResponseValues(stdoutOut, "_full_message")
+				suite.NoError(err, "Failed in [STDOUT]: "+tt.name)
+				suite.Equal(tt.wantMessage, gotMsg, "Failed in [STDOUT]: "+tt.name)
+			}
+			os.Unsetenv("LOG_LEVEL")
+		})
+	}
+}
+
+func Test_getResponseValues(t *testing.T) {
+	type args struct {
+		sourceJson string
+	}
+	tests := []struct {
+		name       string
+		args       args
+		wantGotLvl string
+		wantGotMsg string
+		wantErr    bool
+	}{
+		{
+			name: "Test with json",
+			args: args{
+				sourceJson: `{"level": "debug", "short_message": "hello world"`,
+			},
+			wantGotLvl: "debug",
+			wantGotMsg: "hello world",
+			wantErr:    false,
+		},
+		{
+			name: "Test with json, wrong message field",
+			args: args{
+				sourceJson: `{"level": "debug", "message": "hello world"`,
+			},
+			wantGotLvl: "debug",
+			wantGotMsg: "",
+			wantErr:    true,
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			gotGotLvl, gotGotMsg, err := getResponseValues(tt.args.sourceJson, "short_message")
+			if (err != nil) != tt.wantErr {
+				t.Errorf("getResponseValues() error = %v, wantErr %v", err, tt.wantErr)
+				return
+			}
+			if gotGotLvl != tt.wantGotLvl {
+				t.Errorf("getResponseValues() gotGotLvl = %v, want %v", gotGotLvl, tt.wantGotLvl)
+			}
+			if gotGotMsg != tt.wantGotMsg {
+				t.Errorf("getResponseValues() gotGotMsg = %v, want %v", gotGotMsg, tt.wantGotMsg)
+			}
+		})
+	}
+}
+
+func getResponseValues(sourceJson string, key string) (gotLvl, gotMsg string, err error) {
+	gotLvl, err = jsonparser.GetString([]byte(sourceJson), "level")
+	if err != nil {
+		return
+	}
+	gotMsg, err = jsonparser.GetString([]byte(sourceJson), key)
+	return
+}

main.go

@@ -1,10 +1,12 @@
 package main
 
 import (
+	"strings"
+
 	"github.com/gookit/goutil/envutil"
 	graphql "github.com/lukaszraczylo/go-simple-graphql"
-	libpack_config "github.com/telegram-bot-app/libpack/config"
-	libpack_logging "github.com/telegram-bot-app/libpack/logging"
+	libpack_config "github.com/lukaszraczylo/graphql-monitoring-proxy/config"
+	libpack_logging "github.com/lukaszraczylo/graphql-monitoring-proxy/logging"
 )
 
 var cfg *config
@@ -20,20 +22,44 @@ func parseConfig() {
 	var c config
 	c.Server.PortGraphQL = envutil.GetInt("PORT_GRAPHQL", 8080)
 	c.Server.PortMonitoring = envutil.GetInt("MONITORING_PORT", 9393)
-	c.Server.HostGraphQL = envutil.Getenv("HOST_GRAPHQL", "http://localhost/v1/graphql")
+	c.Server.HostGraphQL = envutil.Getenv("HOST_GRAPHQL", "http://localhost/")
 	c.Client.JWTUserClaimPath = envutil.Getenv("JWT_USER_CLAIM_PATH", "")
 	c.Client.JWTRoleClaimPath = envutil.Getenv("JWT_ROLE_CLAIM_PATH", "")
-	c.Client.JWTRoleRateLimit = envutil.GetBool("JWT_ROLE_RATE_LIMIT", false)
+	c.Client.RoleFromHeader = envutil.Getenv("ROLE_FROM_HEADER", "")
+	c.Client.RoleRateLimit = envutil.GetBool("ROLE_RATE_LIMIT", false)
 	c.Cache.CacheEnable = envutil.GetBool("ENABLE_GLOBAL_CACHE", false)
 	c.Cache.CacheTTL = envutil.GetInt("CACHE_TTL", 60)
 	c.Security.BlockIntrospection = envutil.GetBool("BLOCK_SCHEMA_INTROSPECTION", false)
+	c.Security.IntrospectionAllowed = func() []string {
+		urls := envutil.Getenv("ALLOWED_INTROSPECTION", "")
+		if urls == "" {
+			return nil
+		}
+		return strings.Split(urls, ",")
+	}()
 	c.Logger = libpack_logging.NewLogger()
+	c.Server.HealthcheckGraphQL = envutil.Getenv("HEALTHCHECK_GRAPHQL_URL", "")
 	c.Client.GQLClient = graphql.NewConnection()
-	c.Client.GQLClient.SetEndpoint(c.Server.HostGraphQL)
+	c.Client.GQLClient.SetEndpoint(c.Server.HealthcheckGraphQL)
 	c.Server.AccessLog = envutil.GetBool("ENABLE_ACCESS_LOG", false)
+	c.Server.ReadOnlyMode = envutil.GetBool("READ_ONLY_MODE", false)
+	c.Server.AllowURLs = func() []string {
+		urls := envutil.Getenv("ALLOWED_URLS", "")
+		if urls == "" {
+			return nil
+		}
+		return strings.Split(urls, ",")
+	}()
+	c.Client.ClientTimeout = envutil.GetInt("PROXIED_CLIENT_TIMEOUT", 120)
+	c.Client.FastProxyClient = createFasthttpClient(c.Client.ClientTimeout)
+	c.Server.EnableApi = envutil.GetBool("ENABLE_API", false)
+	c.Server.ApiPort = envutil.GetInt("API_PORT", 9090)
+	c.Api.BannedUsersFile = envutil.Getenv("BANNED_USERS_FILE", "/go/src/app/banned_users.json")
+	c.Server.PurgeOnCrawl = envutil.GetBool("PURGE_METRICS_ON_CRAWL", false)
 	cfg = &c
 	enableCache() // takes close to no resources, but can be used with dynamic query cache
 	loadRatelimitConfig()
+	enableApi()
 }
 
 func main() {

main_test.go

@@ -1,7 +1,6 @@
 package main
 
 import (
-	"fmt"
 	"testing"
 
 	assertions "github.com/stretchr/testify/assert"
@@ -21,14 +20,13 @@ func (suite *Tests) SetupTest() {
 }
 
 func (suite *Tests) BeforeTest(suiteName, testName string) {
-	fmt.Println("BeforeTest")
-	cfg = &config{}
-	parseConfig()
-	StartMonitoringServer()
 }
 
 // func (suite *Tests) AfterTest(suiteName, testName string) {)
 
 func TestSuite(t *testing.T) {
+	cfg = &config{}
+	parseConfig()
+	StartMonitoringServer()
 	suite.Run(t, new(Tests))
 }

monitoring.go

@@ -1,11 +1,11 @@
 package main
 
 import (
-	libpack_monitoring "github.com/telegram-bot-app/libpack/monitoring"
+	libpack_monitoring "github.com/lukaszraczylo/graphql-monitoring-proxy/monitoring"
 )
 
 func StartMonitoringServer() {
-	cfg.Monitoring = libpack_monitoring.NewMonitoring()
+	cfg.Monitoring = libpack_monitoring.NewMonitoring(cfg.Server.PurgeOnCrawl)
 	cfg.Monitoring.AddMetricsPrefix("graphql_proxy")
 	cfg.Monitoring.RegisterDefaultMetrics()
 }

monitoring/defaults.go

@@ -0,0 +1,12 @@
+package libpack_monitoring
+
+func (ms *MetricsSetup) RegisterDefaultMetrics() {
+	ms.RegisterMetricsCounter(MetricsSucceeded, nil)
+	ms.RegisterMetricsCounter(MetricsFailed, nil)
+	ms.RegisterMetricsCounter(MetricsSkipped, nil)
+	ms.RegisterMetricsHistogram(MetricsDuration, nil)
+}
+
+func (ms *MetricsSetup) RegisterGoMetrics() {
+	// TODO: metrics.WriteProcessMetrics(ms.metrics_set)
+}

monitoring/helpers.go

@@ -0,0 +1,63 @@
+package libpack_monitoring
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	libpack_config "github.com/lukaszraczylo/graphql-monitoring-proxy/config"
+)
+
+func (ms *MetricsSetup) get_metrics_name(name string, labels map[string]string) (complete_name string) {
+	var err error
+	if labels == nil {
+		labels = make(map[string]string)
+	}
+	labels["microservice"] = libpack_config.PKG_NAME
+	labels["pod"], err = os.Hostname()
+	if err != nil {
+		labels["pod"] = "unknown"
+	}
+
+	if ms.metrics_prefix != "" {
+		complete_name = ms.metrics_prefix + "_" + name
+	} else {
+		complete_name = name
+	}
+	if labels != nil {
+		complete_name += "{"
+		for k, v := range labels {
+			complete_name += k + "=\"" + v + "\","
+		}
+		complete_name = strings.TrimSuffix(complete_name, ",")
+		complete_name += "}"
+	}
+	return
+}
+
+// validate_metrics_name validates the name of the metric to adhere to the Prometheus naming conventions
+// https://prometheus.io/docs/practices/naming/
+func validate_metrics_name(name string) error {
+	// replace all spaces with underscores and remove all other non-alphanumeric characters
+	name_new := strings.ReplaceAll(name, " ", "_")
+	name_new = strings.Map(func(r rune) rune {
+		if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
+			return r
+		}
+		return -1
+	}, name_new)
+	name_new = strings.ReplaceAll(name_new, "__", "_")
+	name_new = strings.Trim(name_new, "_")
+	if name_new != name {
+		return fmt.Errorf("Invalid metric name: %s, expected %s", name, name_new)
+	}
+	return nil
+}
+
+func compile_metrics_with_labels(name string, labels map[string]string) string {
+	metric_name := name
+	for k, v := range labels {
+		metric_name += "_" + k + "_" + v
+	}
+	return metric_name
+}

monitoring/monitoring.go

@@ -0,0 +1,143 @@
+// Package `libpack_monitoring` provides and easy way to add prometheus metrics to your application.
+// It also provides a way to add custom metrics to the already started prometheus registry.
+
+package libpack_monitoring
+
+import (
+	"fmt"
+	"time"
+
+	"github.com/VictoriaMetrics/metrics"
+	"github.com/gofiber/fiber/v2"
+	"github.com/gookit/goutil/envutil"
+	libpack_config "github.com/lukaszraczylo/graphql-monitoring-proxy/config"
+	logging "github.com/lukaszraczylo/graphql-monitoring-proxy/logging"
+)
+
+type MetricsSetup struct {
+	metrics_prefix     string
+	metrics_set        *metrics.Set
+	metrics_set_custom *metrics.Set
+}
+
+var (
+	log                 *logging.LogConfig
+	purgeMetricsOnCrawl bool
+)
+
+func NewMonitoring(purgeOnCrawl bool) *MetricsSetup {
+	purgeMetricsOnCrawl = purgeOnCrawl
+	log = logging.NewLogger()
+	ms := &MetricsSetup{}
+	ms.metrics_set = metrics.NewSet()
+	ms.metrics_set_custom = metrics.NewSet()
+	go ms.startPrometheusEndpoint()
+	return ms
+}
+
+func (ms *MetricsSetup) startPrometheusEndpoint() {
+	app := fiber.New(fiber.Config{
+		DisableStartupMessage: true,
+		AppName:               fmt.Sprintf("GraphQL Monitoring Proxy - %s v%s", libpack_config.PKG_NAME, libpack_config.PKG_VERSION),
+	})
+	app.Get("/metrics", ms.metricsEndpoint)
+	err := app.Listen(fmt.Sprintf(":%d", envutil.GetInt("MONITORING_PORT", 9393)))
+	if err != nil {
+		fmt.Println("Can't start the service: ", err)
+	}
+}
+
+func (ms *MetricsSetup) metricsEndpoint(c *fiber.Ctx) error {
+	ms.metrics_set.WritePrometheus(c.Response().BodyWriter())
+	ms.metrics_set_custom.WritePrometheus(c.Response().BodyWriter())
+	if purgeMetricsOnCrawl {
+		ms.PurgeMetrics()
+	}
+	return nil
+}
+
+func (ms *MetricsSetup) AddMetricsPrefix(prefix string) {
+	ms.metrics_prefix = prefix
+}
+
+func (ms *MetricsSetup) ListActiveMetrics() []string {
+	return ms.metrics_set.ListMetricNames()
+}
+
+func (ms *MetricsSetup) RegisterMetricsGauge(metric_name string, labels map[string]string, val float64) *metrics.Gauge {
+	if validate_metrics_name(metric_name) != nil {
+		log.Critical("RegisterMetricsGauge() error", map[string]interface{}{"_error": "Invalid metric name", "_metric_name": metric_name})
+		return nil
+	}
+	return ms.metrics_set_custom.GetOrCreateGauge(ms.get_metrics_name(metric_name, labels), func() float64 {
+		// get current value of the gauge and add val to it
+		return val
+	})
+}
+
+func (ms *MetricsSetup) RegisterMetricsCounter(metric_name string, labels map[string]string) *metrics.Counter {
+	if validate_metrics_name(metric_name) != nil {
+		log.Critical("RegisterMetricsCounter() error", map[string]interface{}{"_error": "Invalid metric name", "_metric_name": metric_name})
+		return nil
+	}
+	if metric_name == MetricsSucceeded || metric_name == MetricsFailed || metric_name == MetricsSkipped {
+		return ms.metrics_set.GetOrCreateCounter(ms.get_metrics_name(metric_name, labels))
+	}
+	return ms.metrics_set_custom.GetOrCreateCounter(ms.get_metrics_name(metric_name, labels))
+}
+
+func (ms *MetricsSetup) RegisterFloatCounter(metric_name string, labels map[string]string) *metrics.FloatCounter {
+	if validate_metrics_name(metric_name) != nil {
+		log.Critical("RegisterFloatCounter() error", map[string]interface{}{"_error": "Invalid metric name", "_metric_name": metric_name})
+		return nil
+	}
+	return ms.metrics_set_custom.GetOrCreateFloatCounter(ms.get_metrics_name(metric_name, labels))
+}
+
+func (ms *MetricsSetup) RegisterMetricsSummary(metric_name string, labels map[string]string) *metrics.Summary {
+	if validate_metrics_name(metric_name) != nil {
+		log.Critical("RegisterMetricsSummary() error", map[string]interface{}{"_error": "Invalid metric name", "_metric_name": metric_name})
+		return nil
+	}
+	return ms.metrics_set_custom.GetOrCreateSummary(ms.get_metrics_name(metric_name, labels))
+}
+
+func (ms *MetricsSetup) RegisterMetricsHistogram(metric_name string, labels map[string]string) *metrics.Histogram {
+	if validate_metrics_name(metric_name) != nil {
+		log.Critical("RegisterMetricsHistogram() error", map[string]interface{}{"_error": "Invalid metric name", "_metric_name": metric_name})
+		return nil
+	}
+	return ms.metrics_set_custom.GetOrCreateHistogram(ms.get_metrics_name(metric_name, labels))
+}
+
+func (ms *MetricsSetup) Increment(metric_name string, labels map[string]string) {
+	ms.RegisterMetricsCounter(metric_name, labels).Inc()
+}
+
+func (ms *MetricsSetup) IncrementFloat(metric_name string, labels map[string]string, value float64) {
+	ms.RegisterFloatCounter(metric_name, labels).Add(value)
+}
+
+func (ms *MetricsSetup) Set(metric_name string, labels map[string]string, value uint64) {
+	ms.RegisterMetricsCounter(metric_name, labels).Set(value)
+}
+
+func (ms *MetricsSetup) Update(metric_name string, labels map[string]string, value float64) {
+	ms.RegisterMetricsHistogram(metric_name, labels).Update(value)
+}
+
+func (ms *MetricsSetup) UpdateDuration(metric_name string, labels map[string]string, value time.Time) {
+	ms.RegisterMetricsHistogram(metric_name, labels).UpdateDuration(value)
+}
+
+func (ms *MetricsSetup) UpdateSummary(metric_name string, labels map[string]string, value float64) {
+	ms.RegisterMetricsSummary(metric_name, labels).Update(value)
+}
+
+func (ms *MetricsSetup) RemoveMetrics(metric_name string, labels map[string]string) {
+	ms.metrics_set_custom.UnregisterMetric(ms.get_metrics_name(metric_name, labels))
+}
+
+func (ms *MetricsSetup) PurgeMetrics() {
+	ms.metrics_set_custom.UnregisterAllMetrics()
+}

monitoring/structs.go

@@ -0,0 +1,8 @@
+package libpack_monitoring
+
+const (
+	MetricsSucceeded = "requests_succesful"
+	MetricsFailed    = "requests_failed"
+	MetricsDuration  = "requests_duration"
+	MetricsSkipped   = "requests_skipped"
+)

proxy.go

@@ -2,26 +2,55 @@ package main
 
 import (
 	"crypto/tls"
+	"fmt"
+	"time"
 
 	fiber "github.com/gofiber/fiber/v2"
 	"github.com/gofiber/fiber/v2/middleware/proxy"
-	libpack_monitoring "github.com/telegram-bot-app/libpack/monitoring"
+	libpack_monitoring "github.com/lukaszraczylo/graphql-monitoring-proxy/monitoring"
+	"github.com/valyala/fasthttp"
 )
 
+func createFasthttpClient(timeout int) *fasthttp.Client {
+	return &fasthttp.Client{
+		Name:                     "graphql_proxy",
+		NoDefaultUserAgentHeader: true,
+		TLSConfig: &tls.Config{
+			InsecureSkipVerify: true,
+		},
+		MaxConnsPerHost:               200,
+		ReadTimeout:                   time.Second * time.Duration(timeout),
+		WriteTimeout:                  time.Second * time.Duration(timeout),
+		DisableHeaderNamesNormalizing: true,
+	}
+}
+
 func proxyTheRequest(c *fiber.Ctx) error {
+	if !checkAllowedURLs(c) {
+		cfg.Logger.Error("Request blocked", map[string]interface{}{"path": c.Path()})
+		cfg.Monitoring.Increment(libpack_monitoring.MetricsSkipped, nil)
+		c.Status(403).SendString("Request blocked - not allowed URL")
+		return nil
+	}
+	c.Request().Header.DisableNormalizing()
 	c.Request().Header.Add("X-Real-IP", c.IP())
-	c.Request().Header.Add("X-Forwarded-For", c.IP())
+	c.Request().Header.Add(fiber.HeaderXForwardedFor, string(c.Request().Header.Peek("X-Forwarded-For")))
 
-	proxy.WithTlsConfig(&tls.Config{
-		InsecureSkipVerify: true,
-	})
+	proxy.WithClient(cfg.Client.FastProxyClient)
 
-	err := proxy.DoRedirects(c, cfg.Server.HostGraphQL, 3)
+	cfg.Logger.Debug("Proxying the request", map[string]interface{}{"path": c.Path(), "body": string(c.Request().Body()), "headers": c.GetReqHeaders(), "request_uuid": c.Locals("request_uuid")})
+	err := proxy.DoRedirects(c, cfg.Server.HostGraphQL+c.Path(), 3)
 	if err != nil {
 		cfg.Logger.Error("Can't proxy the request", map[string]interface{}{"error": err.Error()})
 		cfg.Monitoring.Increment(libpack_monitoring.MetricsFailed, nil)
 		return err
 	}
+	cfg.Logger.Debug("Received proxied response", map[string]interface{}{"path": c.Path(), "response_body": string(c.Response().Body()), "response_code": c.Response().StatusCode(), "headers": c.GetRespHeaders(), "request_uuid": c.Locals("request_uuid")})
+
+	if c.Response().StatusCode() != 200 {
+		cfg.Monitoring.Increment(libpack_monitoring.MetricsFailed, nil)
+		return fmt.Errorf("Received non-200 response from the GraphQL server: %d", c.Response().StatusCode())
+	}
 
 	c.Response().Header.Del(fiber.HeaderServer)
 	return nil

ratelimit.go

@@ -32,10 +32,10 @@ func loadRatelimitConfig() error {
 		if err == nil {
 			return nil
 		}
-		cfg.Logger.Error("Failed to load config", map[string]interface{}{"path": path, "error": err})
+		cfg.Logger.Debug("Failed to load config", map[string]interface{}{"path": path, "error": err})
 	}
 
-	cfg.Logger.Debug("Rate limit config not found")
+	cfg.Logger.Error("Rate limit config not found")
 	return os.ErrNotExist
 }
 

server.go

@@ -2,43 +2,79 @@ package main
 
 import (
 	"fmt"
+	"strconv"
 	"time"
 
 	fiber "github.com/gofiber/fiber/v2"
 	"github.com/gofiber/fiber/v2/middleware/cors"
+	"github.com/google/uuid"
 
 	jsoniter "github.com/json-iterator/go"
-	libpack_monitoring "github.com/telegram-bot-app/libpack/monitoring"
+	libpack_config "github.com/lukaszraczylo/graphql-monitoring-proxy/config"
+	libpack_monitoring "github.com/lukaszraczylo/graphql-monitoring-proxy/monitoring"
 )
 
 var json = jsoniter.ConfigCompatibleWithStandardLibrary
 
 // StartHTTPProxy starts the HTTP and points it to the GraphQL server.
 func StartHTTPProxy() {
-	server := fiber.New()
+	server := fiber.New(fiber.Config{
+		DisableStartupMessage: true,
+		AppName:               fmt.Sprintf("GraphQL Monitoring Proxy - %s v%s", libpack_config.PKG_NAME, libpack_config.PKG_VERSION),
+	})
 
 	server.Use(cors.New(cors.Config{
 		AllowOrigins: "*",
 	}))
 
-	server.Post("/v1/graphql", processGraphQLRequest)
+	// add middleware to check if the request is a GraphQL query
+	server.Use(AddRequestUUID)
 
 	server.Get("/healthz", healthCheck)
+	server.Get("/livez", healthCheck)
+
+	server.Post("/*", processGraphQLRequest)
+	server.Get("/*", proxyTheRequest)
+
+	cfg.Logger.Info("GraphQL query proxy started", map[string]interface{}{"port": cfg.Server.PortGraphQL})
 	err := server.Listen(fmt.Sprintf(":%d", cfg.Server.PortGraphQL))
 	if err != nil {
 		cfg.Logger.Critical("Can't start the service", map[string]interface{}{"error": err.Error()})
 	}
 }
 
+func AddRequestUUID(c *fiber.Ctx) error {
+	c.Locals("request_uuid", uuid.NewString())
+	return c.Next()
+}
+
+func checkAllowedURLs(c *fiber.Ctx) bool {
+	if len(cfg.Server.AllowURLs) == 0 {
+		return true
+	}
+	for _, allowedURL := range cfg.Server.AllowURLs {
+		if c.Path() == allowedURL {
+			return true
+		}
+	}
+	return false
+}
+
 func healthCheck(c *fiber.Ctx) error {
-	// query := `{ __typename }`
-	// _, err := cfg.Client.GQLClient.Query(query, nil, nil)
-	// if err != nil {
-	// 	cfg.Logger.Error("Can't reach the GraphQL server", map[string]interface{}{"error": err.Error()})
-	// 	cfg.Monitoring.Increment(libpack_monitoring.MetricsFailed, nil)
-	// 	return c.SendStatus(500)
-	// }
-	return c.SendStatus(200)
+	if len(cfg.Server.HealthcheckGraphQL) > 0 {
+		cfg.Logger.Debug("Health check enabled", map[string]interface{}{"url": cfg.Server.HealthcheckGraphQL})
+		query := `{ __typename }`
+		_, err := cfg.Client.GQLClient.Query(query, nil, nil)
+		if err != nil {
+			cfg.Logger.Error("Can't reach the GraphQL server", map[string]interface{}{"error": err.Error()})
+			cfg.Monitoring.Increment(libpack_monitoring.MetricsFailed, nil)
+			c.Status(500).SendString("Can't reach the GraphQL server with {__typename} query")
+			return err
+		}
+	}
+	cfg.Logger.Debug("Health check returning OK")
+	c.Status(200).SendString("Health check OK")
+	return nil
 }
 
 func processGraphQLRequest(c *fiber.Ctx) error {
@@ -54,8 +90,19 @@ func processGraphQLRequest(c *fiber.Ctx) error {
 		extractedUserID, extractedRoleName = extractClaimsFromJWTHeader(string(authorization))
 	}
 
+	if checkIfUserIsBanned(c, extractedUserID) {
+		return nil
+	}
+
+	if len(cfg.Client.RoleFromHeader) > 0 {
+		extractedRoleName = string(c.Request().Header.Peek(cfg.Client.RoleFromHeader))
+		if extractedRoleName == "" {
+			extractedRoleName = "-"
+		}
+	}
+
 	// Implementing rate limiting if enabled
-	if cfg.Client.JWTRoleRateLimit {
+	if cfg.Client.RoleRateLimit {
 		cfg.Logger.Debug("Rate limiting enabled", map[string]interface{}{"user_id": extractedUserID, "role_name": extractedRoleName})
 		if !rateLimitedRequest(extractedUserID, extractedRoleName) {
 			c.Status(429).SendString("Rate limit exceeded, try again later")
@@ -63,11 +110,30 @@ func processGraphQLRequest(c *fiber.Ctx) error {
 		}
 	}
 
-	opType, opName, cacheFromQuery, shouldBlock := parseGraphQLQuery(c)
+	opType, opName, cacheFromQuery, cache_time, shouldBlock, should_ignore := parseGraphQLQuery(c)
 	if shouldBlock {
 		return nil
 	}
 
+	if should_ignore {
+		cfg.Logger.Debug("Request passed as-is - not a GraphQL")
+		return proxyTheRequest(c)
+	}
+
+	if cache_time > 0 {
+		cfg.Logger.Debug("Cache time set via query", map[string]interface{}{"cache_time": cache_time})
+		cache_time = cfg.Cache.CacheTTL
+	}
+
+	if cache_time == 0 && !cacheFromQuery {
+		cacheQuery := c.Request().Header.Peek("X-Cache-Graphql-Query")
+		if cacheQuery != nil {
+			cache_time, _ = strconv.Atoi(string(cacheQuery))
+			cfg.Logger.Debug("Cache time set via header", map[string]interface{}{"cache_time": cache_time})
+			cacheFromQuery = true
+		}
+	}
+
 	wasCached := false
 
 	// Handling Cache Logic
@@ -76,12 +142,12 @@ func processGraphQLRequest(c *fiber.Ctx) error {
 		queryCacheHash = calculateHash(c)
 
 		if cachedResponse := cacheLookup(queryCacheHash); cachedResponse != nil {
-			cfg.Logger.Debug("Cache hit", map[string]interface{}{"hash": queryCacheHash, "user_id": extractedUserID})
+			cfg.Logger.Debug("Cache hit", map[string]interface{}{"hash": queryCacheHash, "user_id": extractedUserID, "request_uuid": c.Locals("request_uuid")})
 			c.Send(cachedResponse)
 			wasCached = true
 		} else {
-			cfg.Logger.Debug("Cache miss", map[string]interface{}{"hash": queryCacheHash, "user_id": extractedUserID})
-			proxyAndCacheTheRequest(c, queryCacheHash)
+			cfg.Logger.Debug("Cache miss", map[string]interface{}{"hash": queryCacheHash, "user_id": extractedUserID, "request_uuid": c.Locals("request_uuid")})
+			proxyAndCacheTheRequest(c, queryCacheHash, cache_time)
 		}
 	} else {
 		proxyTheRequest(c)
@@ -96,9 +162,15 @@ func processGraphQLRequest(c *fiber.Ctx) error {
 }
 
 // Additional helper function to avoid code repetition
-func proxyAndCacheTheRequest(c *fiber.Ctx, queryCacheHash string) {
-	proxyTheRequest(c)
-	cfg.Cache.CacheClient.Set(queryCacheHash, c.Response().Body(), time.Duration(cfg.Cache.CacheTTL)*time.Second)
+func proxyAndCacheTheRequest(c *fiber.Ctx, queryCacheHash string, cache_time int) {
+	err := proxyTheRequest(c)
+	if err != nil {
+		cfg.Logger.Error("Can't proxy the request", map[string]interface{}{"error": err.Error()})
+		cfg.Monitoring.Increment(libpack_monitoring.MetricsFailed, nil)
+		c.Status(500).SendString("Can't proxy the request - try again later")
+		return
+	}
+	cfg.Cache.CacheClient.Set(queryCacheHash, c.Response().Body(), time.Duration(cache_time)*time.Second)
 	c.Send(c.Response().Body())
 }
 
@@ -112,12 +184,14 @@ func logAndMonitorRequest(c *fiber.Ctx, userID, opType, opName string, wasCached
 
 	if cfg.Server.AccessLog {
 		cfg.Logger.Info("Request processed", map[string]interface{}{
-			"ip":      c.IP(),
-			"user_id": userID,
-			"op_type": opType,
-			"op_name": opName,
-			"time":    duration,
-			"cache":   wasCached,
+			"ip":           c.IP(),
+			"fwd-ip":       string(c.Request().Header.Peek("X-Forwarded-For")),
+			"user_id":      userID,
+			"op_type":      opType,
+			"op_name":      opName,
+			"time":         duration,
+			"cache":        wasCached,
+			"request_uuid": c.Locals("request_uuid"),
 		})
 	}
 

static/kubernetes-single-deployment.yaml

@@ -0,0 +1,121 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: hasura-w-proxy-internal
+  labels:
+    app: hasura-w-proxy-internal
+    type: support
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app: hasura-w-proxy-internal
+      type: support
+  template:
+    metadata:
+      labels:
+        app: hasura-w-proxy-internal
+        type: support
+    spec:
+      securityContext:
+        runAsUser: 65534 # nobody
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: node-role.kubernetes.io/worker
+                operator: Exists
+      containers:
+      - name: hasura
+        image: hasura/graphql-engine:v2.33.1-ce
+        ports:
+          - name: hasura-internal
+            containerPort: 8080
+        livenessProbe:
+          httpGet:
+            path: /healthz
+            port: 8080
+          initialDelaySeconds: 30
+        resources:
+          limits:
+            cpu: "1"
+            memory: "640Mi"
+          requests:
+            cpu: "0.75"
+            memory: "512Mi"
+        env:
+          - name: HASURA_GRAPHQL_DATABASE_URL
+            value: postgres://postgres:xxx@yyy:5432/postgres
+          - name: HASURA_GRAPHQL_ENABLE_CONSOLE
+            value: "true"
+          - name: HASURA_GRAPHQL_DEV_MODE
+            value: "true"
+          - name: HASURA_GRAPHQL_ENABLE_TELEMETRY
+            value: "false"
+          - name: HASURA_GRAPHQL_EXPERIMENTAL_FEATURES
+            value: "inherited_roles"
+          - name: HASURA_GRAPHQL_PG_CONNECTIONS
+            value: "20"
+          - name: HASURA_GRAPHQL_LOG_LEVEL
+            value: "error"
+      - name: graphql-proxy
+        image: ghcr.io/lukaszraczylo/graphql-monitoring-proxy:latest
+        imagePullPolicy: Always
+        resources:
+          limits:
+            cpu: "1"
+            memory: "640Mi"
+          requests:
+            cpu: "0.75"
+            memory: "128Mi"
+        livenessProbe:
+          httpGet:
+            path: /healthz
+            port: 8080
+          initialDelaySeconds: 5
+          timeoutSeconds: 5
+        ports:
+          - name: web
+            containerPort: 8181
+          - name: monitoring
+            containerPort: 9393
+        env:
+          - name: PORT_GRAPHQL
+            value: "8181"
+          - name: MONITORING_PORT
+            value: "9393"
+          - name: HOST_GRAPHQL
+            value: http://localhost:8080/
+          - name: ENABLE_GLOBAL_CACHE
+            value: "true"
+          - name: CACHE_TTL
+            value: "10"
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: hasura-w-proxy-internal
+  labels:
+    app: hasura-w-proxy-internal
+    type: support
+  annotations:
+    prometheus.io/scrape: "true"
+    prometheus.io/port: "9393"
+    prometheus.io/path: "/metrics"
+spec:
+  ports:
+  - name: hasura
+    port: 8080
+    targetPort: 8080
+  - name: proxy
+    port: 8181
+    targetPort: 8181
+  - name: monitoring
+    port: 9393
+    targetPort: 9393
+  selector:
+    app: hasura-w-proxy-internal
+    type: support
+  type: ClusterIP

struct_config.go

@@ -1,10 +1,11 @@
 package main
 
 import (
-	"github.com/akyoto/cache"
 	graphql "github.com/lukaszraczylo/go-simple-graphql"
-	libpack_logging "github.com/telegram-bot-app/libpack/logging"
-	libpack_monitoring "github.com/telegram-bot-app/libpack/monitoring"
+	libpack_cache "github.com/lukaszraczylo/graphql-monitoring-proxy/cache"
+	libpack_logging "github.com/lukaszraczylo/graphql-monitoring-proxy/logging"
+	libpack_monitoring "github.com/lukaszraczylo/graphql-monitoring-proxy/monitoring"
+	"github.com/valyala/fasthttp"
 )
 
 // config is a struct that holds the configuration of the application.
@@ -14,26 +15,41 @@ type config struct {
 
 	// Server holds the configuration of the server _ONLY_.
 	Server struct {
-		PortGraphQL    int
-		PortMonitoring int
-		HostGraphQL    string
-		AccessLog      bool
+		PortGraphQL        int
+		PortMonitoring     int
+		HostGraphQL        string
+		HealthcheckGraphQL string
+		AccessLog          bool
+		ReadOnlyMode       bool
+		AllowURLs          []string
+		EnableApi          bool
+		ApiPort            int
+		PurgeOnCrawl       bool
 	}
 
 	Client struct {
 		JWTUserClaimPath string
 		JWTRoleClaimPath string
-		JWTRoleRateLimit bool
+		RoleRateLimit    bool
+		RoleFromHeader   string
 		GQLClient        *graphql.BaseClient
+		FastProxyClient  *fasthttp.Client
+		proxy            string
+		ClientTimeout    int
 	}
 
 	Cache struct {
 		CacheEnable bool
 		CacheTTL    int
-		CacheClient *cache.Cache
+		CacheClient *libpack_cache.Cache
+	}
+
+	Api struct {
+		BannedUsersFile string
 	}
 
 	Security struct {
-		BlockIntrospection bool
+		BlockIntrospection   bool
+		IntrospectionAllowed []string
 	}
 }