In case you're stuck with a file upload problem in one of your app deployed in Kubernetes, this blog may help you.

Just a brief background:

  • We have an app built in Phoenix (Elixir) and is deployed in Kubernetes
  • We used nginx as the ingress controller
  • We have a backend nginx-service that connects to our pods

Like most apps, there will always be some instance that you need to provide a file upload feature to upload pics, files, etc. Nginx and Kubernetes have default limits in the file upload which you will encounter.

413 Request Entity Too Large

In order to fix this, you need to do the following:

  • Set the client-max-body-size to your desired limit in your nginx .conf files.

Ex.

server {
  listen 80;
  server_name domain.com;
  client_max_body_size 50M;
  • Set the client-max-body-size in your ingress config
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    kubernetes.io/tls-acme: "true"
    kubernetes.io/ingress.class: "nginx"
    ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.org/proxy-connect-timeout: "30s"
    nginx.org/proxy-read-timeout: "20s"
    nginx.org/client-max-body-size: "50m"
  • If you want to apply this config all over your nginx-ingress controller, you can set it up in the configmap.yml
apiVersion: v1
data:
  proxy-connect-timeout: "15"
  proxy-read-timeout: "600"
  proxy-send-timeout: "600"
  hsts-include-subdomains: "false"
  body-size: "64m"
  server-name-hash-bucket-size: "256"
  client-max-body-size: "50m"
kind: ConfigMap
metadata:
  namespace: nginx-ingress
  name: nginx

502 Bad Gateway

This is a wildcard issue. In our experience, our pods gets terminated when we increased the file upload limit. We look on all sides of our app to investigate this. In the end, we found that the resource limits of our containers is the cause of this issue.

spec:
       containers:
         - name: app
           resources:
             requests:
               memory: "64Mi"
               cpu: "250m"
             limits:
               memory: "128Mi"
               cpu: "500m"

By increasing the limits, we were able to solve the problem.