Stories Tag: minio

How to setup Distributed MinIO Cluster on Kubernetes | by Faraz Peerbaksh | Dec, 2020 | Medium

admin 4 weeks ago in

MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service. Designed to be Kubernetes Native. MinIO runs on bare metal, network attached storage and every public cloud. Distributed MinIO provides protection against multiple node/drive failures and bit rot using erasure code. As the minimum disks required for distributed MinIO is 4 (same as…

Kubernetes 클러스터에 private docker registry 설정하기 (MinIO 포함) | by Jung-taek Lim | Dec, 2020 | Medium

admin 4 weeks ago in

위의 문서를 통해 k3s 클러스터의 기본 설정이 완료되었다. 추가 설정 없이 docker hub 를 통해 이미지를 pull 하고 push 할 수 있지만, 여러 제약도 있고 리모트 서버에 읽고 써야 되어서 아무래도 빠릿빠릿한 부분이 덜하다. 그래서 많은 경우 private docker registry 를 구성하고 이를 사용하도록 하는 듯 하다. docker registry 를 구성하려면 역시 스토리지를 설정해야 하는데, docker registry helm chart 에서 S3 에 저장하는 기능을 제공한다….

Spark 3.0.0 history server with minIO | by Suchit Gupta | Oct, 2020 | Medium

admin 3 months ago in

Step 1: Enable K8 and set up Dashboard on local. Step 2: Set up the spark google operator Step 3: Get the minIO in the docker-compose and push the stack docker-compose.yaml Command to push the stack: After the push, UI should be accessible at http://localhost:9000/minio/ Step 4: Set up the spark history server Dockerfile: spark-defaults.conf Build the docker image Create secret for S3 Install stable…

MinIO S3 Storage Proxy in AKS – Sebastian Nyberg – Medium

admin 8 months ago in

Why give up all those S3 tools because you are using Azure? The S3 API has become more or less a standard interface for cloud storage. Whether you’re using AWS, GCP, IBM Cloud, DigitalOcean, well, pretty much any cloud provider except for Azure, storage is provided with a S3-compliant API. For this reason, it is quite common that tools within the Open Source community does…

Airflow kubernetes executor & persistent volume – Ibnu Bayhaqi – Medium

admin 8 months ago in

Right now I will try to use kubernetes executor for replacing celery executor in Airflow. And in the future I will always use kubernetes executor, for more flexible for the count of node “worker” and hopefully can reduce cost. 1 node that always running is for webserver, scheduler, minio server it takes no more than 4GB ram (for my case). To change Airflow executor to…