background
MinIO is the world's leading object storage pioneer, with read / write speeds of 183 GB / s and 171 GB / s on standard hardware. MinIO is used as the main storage of cloud native applications. Compared with traditional object storage, cloud native applications need higher throughput and lower latency. By adding more clusters, you can expand the namespace and more racks until you achieve your goal. At the same time, it conforms to the architecture and construction process of all native cloud computing, and includes the latest new technologies and concepts of cloud computing.
As for object storage, it is nothing more than file upload, download and deletion, plus bucket operation. Here we use SpringBoot to integrate MinIO single instance combat. For the high availability and scalability test of MinIO distributed cluster, please refer to the article: High availability test of MinIO distributed file storage cluster developed by full stack
- Barrel management;
- Object management (upload, download, delete);
- Object pre signature;
- Bucket strategy management;
rely on
Note: I use 7 X version for experiment and demonstration, the latest version of 8 The MinIO background management interface of X is different, but after our actual production test, the interfaces are compatible.
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>7.1.4</version>
</dependency>
MinIO runs under a single instance of Docker
docker run -p 9000:9000 \ --name minio1 \ -v /opt/minio/data-single \ -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" \ -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \ minio/minio server /data
Interface test
Based on MinIO client, it mainly realizes service interfaces such as bucket management, object management and object pre signature.
Provide external operation interfaces for file upload, download and deletion with RESTful API;
Upload, download and delete test files using PostMan interface: header information: content type: multipart / form data
# upload curl --location --request POST 'localhost:8090/minio/uploadFile' \ --header 'Content-Type: multipart/form-data' \ --form 'file=@"/C:/Users/nxq01/Downloads/springboot-minio-master.zip"' # download curl --location --request POST 'localhost:8090/minio/downloadFile' \ --form 'bucketName="heartsuit"' \ --form 'originalName="springboot-minio-master.zip"' \ --form 'filePath="2021-11-23/92cf3f69-501b-41de-83ae-f67e5a57f35f.zip"' # delete curl --location --request POST 'localhost:8090/minio/deleteFile' \ --header 'Content-Type: multipart/form-data' \ --form 'bucketName="heartsuit"' \ --form 'filePath="2021-11-23/92cf3f69-501b-41de-83ae-f67e5a57f35f.zip"'
Core interface
- Controller
@RequestMapping("/minio")
@RestController
@Slf4j
public class MinIOController {
@Autowired
private MinIOService minIOService;
@Autowired
private MinIOConfig minioConfig;
@PostMapping("/uploadFile")
public Result<String> uploadFile(MultipartFile file, String bucketName) {
try {
bucketName = !StringUtils.isEmpty(bucketName) ? bucketName : minioConfig.getBucketName();
if (!minIOService.bucketExists(bucketName)) {
minIOService.makeBucket(bucketName);
}
String fileName = file.getOriginalFilename();
assert fileName != null;
// According to the business design, set the storage path: create a directory by day
String objectName = new SimpleDateFormat("yyyy-MM-dd/").format(new Date())
+ UUID.randomUUID().toString()
+ fileName.substring(fileName.lastIndexOf("."));
minIOService.putObject(bucketName, file, objectName);
log.info("The file format is:{}", file.getContentType());
log.info("The original name of the file is:{}", fileName);
log.info("The bucket of file storage is:{}", bucketName);
log.info("File object path is:{}", objectName);
return Result.success(minIOService.getObjectUrl(bucketName, objectName));
} catch (Exception e) {
e.printStackTrace();
return Result.error("Upload failed");
}
}
@PostMapping("/deleteFile")
public Result<String> deleteFile(String bucketName, String filePath) throws Exception {
if (!minIOService.bucketExists(bucketName)) {
throw new Exception("The bucket does not exist");
}
boolean status = minIOService.removeObject(bucketName, filePath);
return status ? Result.success("Deleted successfully") : Result.success("Delete failed");
}
@PostMapping("/downloadFile")
public Result<String> downloadFile(String bucketName, String filePath, String originalName, HttpServletResponse response) throws Exception {
if (!minIOService.bucketExists(bucketName)) {
throw new Exception("The bucket does not exist");
}
boolean status = minIOService.downloadFile(bucketName, filePath, originalName, response);
return status ? Result.success("Download successful") : Result.success("Download failed");
}
}
- Service
@Service
public class MinIOServiceImpl implements MinIOService {
@Autowired
private MinIOUtils minIOUtils;
/**
* Determine whether the bucket exists
*
* @param bucketName
* @return
*/
@Override
public boolean bucketExists(String bucketName) {
return minIOUtils.bucketExists(bucketName);
}
/**
* Create bucket
*
* @param bucketName
*/
@Override
public void makeBucket(String bucketName) {
minIOUtils.makeBucket(bucketName);
}
/**
* File upload
*
* @param bucketName
* @param objectName
* @param filename
*/
@Override
public void putObject(String bucketName, String objectName, String filename) {
minIOUtils.putObject(bucketName, objectName, filename);
}
@Override
public void putObject(String bucketName, String objectName, InputStream stream, String contentType) {
minIOUtils.putObject(bucketName, objectName, stream, contentType);
}
/**
* File upload
*
* @param bucketName
* @param multipartFile
*/
@Override
public void putObject(String bucketName, MultipartFile multipartFile, String filename) {
minIOUtils.putObject(bucketName, multipartFile, filename);
}
/**
* Delete file
* @param bucketName
* @param filePath
*/
@Override
public boolean removeObject(String bucketName,String filePath) {
return minIOUtils.removeObject(bucketName,filePath);
}
/**
* Download File
* @param bucketName
* @param filePath
* @param originalName
* @param response
* @return
*/
@Override
public boolean downloadFile(String bucketName, String filePath, String originalName, HttpServletResponse response) {
return minIOUtils.downloadFile(bucketName,filePath, originalName, response);
}
/**
* Get file path
* @param bucketName
* @param objectName
* @return
*/
@Override
public String getObjectUrl(String bucketName,String objectName) {
return minIOUtils.getObjectUrl(bucketName,objectName);
}
}
The miniutils tool class is omitted. Please refer to the project source code at the end of the article.
Possible problems
- SpringBoot upload file error: org springframework. web. multipart. MaxUploadSizeExceededException: Maximum upload size exceeded; nested exception is java. lang. IllegalStateException: org. apache. tomcat. util. http. fileupload. impl. FileSizeLimitExceededException: The field file exceeds its maximum permitted size of 1048576 bytes.
This is because the SpringBoot project limits the size of uploaded files, which is 1M by default. When a user uploads a file with a size of more than 1M, the above error will be thrown, which can be modified through the following configuration.
spring:
servlet:
multipart:
maxFileSize: 10MB
maxRequestSize: 30MB
- Unable to enter the container through docker exec -it cd34c345960c /bin/bash, with an error: OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused “exec: “/”: permission denied”: unknown
Solution: change docker exec -it cd34c345960c /bin/bash to: docker exec -it cd34c345960c sh or: docker exec -it cd34c345960c /bin/sh
Source Code
See GitHub for complete source code: https://github.com/heartsuit/demo-spring-boot/tree/master/springboot-minio
Reference
If you have any questions or any bugs are found, please feel free to contact me.
Your comments and suggestions are welcome!