catalogue
What are the common IO models?
Two methods commonly used for keyboard input
How many types of IO streams are there in Java?
Since there is a byte stream, why should there be a character stream?
Several ways of reading and writing large files
Note: This article refers to docs/java/basis/io.md · SnailClimb/JavaGuide - Gitee.com
Java File file class summary, read this article is enough (I) - know
Several ways and tests of reading and writing large files in Java IO_ Weeds - CSDN blog
What is I/O?
I/O (Input/Output) is Input/Output.
Let's first interpret I/O from the perspective of computer structure.
According to Feng Neumann structure, computer structure is divided into five parts: arithmetic unit, controller, memory, input device and output device.
Input devices (such as keyboards) and output devices (such as displays) are external devices. Network card and hard disk can belong to either input device or output device.
The input device inputs data to the computer, and the output device receives the data output by the computer.
From the perspective of computer structure, I/O describes the process of communication between computer system and external devices.
Let's first interpret I/O from the perspective of application.
According to the operating system related knowledge learned in the University: in order to ensure the stability and security of the operating system, the address space of a process is divided into User space and Kernel space.
For example, the applications we usually run run run in user space. Only the kernel space can carry out resource related operations at the system state level, such as file management, process communication, memory management and so on. In other words, if we want to perform IO operations, we must rely on the ability of kernel space.
Moreover, programs in user space cannot directly access kernel space.
When you want to perform IO operations, because you do not have permission to perform these operations, you can only initiate a system call and request the operating system to help complete them.
Therefore, if the user process wants to perform IO operations, it must indirectly access the kernel space through {system call}
In the ordinary development process, we are most exposed to disk IO (read and write files) and network IO (network request and response).
From the perspective of application, our application initiates IO calls (system calls) to the kernel of the operating system, and the kernel in charge of the operating system performs specific IO operations. In other words, our application actually just initiates the call of IO operation, and the specific IO execution is completed by the kernel of the operating system.
When an application initiates an I/O call, it goes through two steps:
1. The kernel waits for the I/O device to prepare data
2 the kernel copies data from kernel space to user space.
What are the common IO models?
Under UNIX system, there are five IO models: synchronous blocking I/O, synchronous non blocking I/O, I/O multiplexing, signal driven I/O # and asynchronous I/O.
These are also the five IO models we often mention.
Two methods commonly used for keyboard input
Method 1: Pass Scanner Scanner input = new Scanner(System.in); String s = input.nextLine(); input.close(); Method 2: Pass BufferedReader BufferedReader input = new BufferedReader(new InputStreamReader(System.in)); String s = input.readLine();
How many types of IO streams are there in Java?
According to the flow direction, it can be divided into input flow and output flow;
According to the division of operation units, it can be divided into byte stream and character stream;
According to the role of flow, it is divided into node flow and processing flow.
The Java IO stream involves more than 40 classes. These classes look very messy, but they are actually very regular, and there is a very close relationship between them. The more than 40 classes of the Java IO stream are derived from the following four abstract classes and base classes.
InputStream/Reader: the base class of all input streams. The former is byte input stream and the latter is character input stream.
OutputStream/Writer: the base class of all output streams. The former is byte output stream and the latter is character output stream.
Structure diagram classified by operation mode:
Classification structure chart by operation object:
Since there is a byte stream, why should there be a character stream?
The essence of the problem is to ask: whether it is file reading and writing or network sending and receiving, the smallest storage unit of information is bytes. Why should I/O stream operation be divided into byte stream operation and character stream operation?
Answer: the character stream is obtained by converting bytes by the Java virtual machine. The problem is that this process is very time-consuming. Moreover, if we don't know the encoding type, it is easy to have the problem of garbled code. Therefore, I/O flow simply provides an interface for directly operating characters, which is convenient for us to stream characters at ordinary times. It is better to use byte stream for audio files, pictures and other media files. If characters are involved, it is better to use character stream.
File class
java.io.File is an important class of files and directories
The file includes the contents of the file and the basic attributes of the file
Basic file attributes: name, size, extension, modification time, etc
File does not involve specific file content, only attributes!!!
createNewFile, delete, exits, getAbsolutePath, getName, getParent, getPath, isDirectory, isFile, length, listFiles, mkdir, mkdirs...
public class Main {
public static void main(String[] args)
{
String path = "/users/qjq/IdeaProjects"; //Directory path already exists
// 1. Create a directory
File d = new File(path + "/testFile");
if (!d.exists()) { // exits determines whether the File object exists
d.mkdir(); // Mkdirs creates single level directories, and mkdirs creates multi-level directories continuously
}
System.out.println("Is d directory? " + d.isDirectory()); // Is isDirectory a directory
// 2. Create files
File f = new File(path + "/testFile/abc.txt");
if (!f.exists())
{
try {
f.createNewFile(); // Create a new file (not a directory)
} catch (IOException e) {
e.printStackTrace(); // Errors may be reported due to insufficient permissions or full disk
}
}
// 3. Output file related attributes
System.out.println("Is f file? " + f.isFile());
System.out.println("Name: " + f.getName());
System.out.println("Path: " + f.getPath());
System.out.println("Parent: " + f.getParent());
System.out.println("Size: " + f.length() + " bytes");
System.out.println("Last Modified time: " + f.lastModified() + " ms");
// 4. All file information in the convenience d directory
System.out.println("list files in d directory");
File[] fs = d.listFiles(); // List all the sub files in the d directory, excluding the files in the sub directory
for (File f1 : fs) {
if (f1.isDirectory()) {
// TO DO WITH subdirectory
}
System.out.println(f1.getPath());
}
// 5. Delete this file
f.delete(); // delete deletes a file or directory
// 6. Delete directory
d.delete();
/**
* Output:
Is d directory? true
Is f file? true
Name: abc.txt
Path: /users/qjq/IdeaProjects/testFile/abc.txt
Parent: /users/qjq/IdeaProjects/testFile
Size: 0 bytes
Last Modified time: 1571230583000 ms
list files in d directory
/users/qjq/IdeaProjects/testFile/abc.txt
*/
}
}Several ways of reading and writing large files
First, OldIO:
public static void oldIOReadFile() throws IOException{
BufferedReader br = new BufferedReader(new FileReader("G://lily_947.txt"));
PrintWriter pw = new PrintWriter("G://oldIO.tmp");
char[] c = new char[100*1024*1024];
for(;;){
if(br.read(c)!=-1){
pw.print(c);
}else{
break;
}
}
pw.close();
br.close();
} It takes 70.79s
Second, newIO:
public static void newIOReadFile() throws IOException{
FileChannel read = new RandomAccessFile("G://lily_947.txt","r").getChannel();
FileChannel writer = new RandomAccessFile("G://newIO.tmp","rw").getChannel();
ByteBuffer bb = ByteBuffer.allocate(200*1024*1024);
while(read.read(bb)!=-1){
bb.flip();
writer.write(bb);
bb.clear();
}
read.close();
writer.close();
} It takes 47.24s
Third, RandomAccessFile:
public static void randomReadFile() throws IOException{
RandomAccessFile read = new RandomAccessFile("G://lily_947.txt","r");
RandomAccessFile writer = new RandomAccessFile("G://random.tmp","rw");
byte[] b = new byte[200*1024*1024];
while(read.read(b)!=-1){
writer.write(b);
}
writer.close();
read.close();
}
Time consuming 46.65
Fourth, MappedByteBuffer:
public static void mappedBuffer() throws IOException{
FileChannel read = new FileInputStream("G://lily_947.txt").getChannel();
FileChannel writer = new RandomAccessFile("G://buffer.tmp","rw").getChannel();
long i = 0;
long size = read.size()/30;
ByteBuffer bb,cc = null;
while(i<read.size()&&(read.size()-i)>size){
bb = read.map(FileChannel.MapMode.READ_ONLY, i, size);
cc = writer.map(FileChannel.MapMode.READ_WRITE, i, size);
cc.put(bb);
i+=size;
bb.clear();
cc.clear();
}
bb = read.map(FileChannel.MapMode.READ_ONLY, i, read.size()-i);
cc.put(bb);
bb.clear();
cc.clear();
read.close();
writer.close();
} Compared with the last direct memory mapping method, the previous test is actually meaningless, basically second kill.....
For large files, there will be insufficient memory during direct block mapping, which is caused by the fact that the MappedByteBuffer is not released. sun does not provide a method to directly recycle the MappedByteBuffer area. At this time, there are two methods to solve it. The first one is stupid:
System.gc();
System.runFinalization();
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
The second method is to call the clean method through reflection
public static void unmap(final MappedByteBuffer buffer) {
if (buffer == null) {
return;
}
AccessController.doPrivileged(new PrivilegedAction<Object>() {
public Object run() {
try {
Method getCleanerMethod = buffer.getClass().getMethod("cleaner", new Class[0]);
if (getCleanerMethod != null) {
getCleanerMethod.setAccessible(true);
Object cleaner = getCleanerMethod.invoke(buffer, new Object[0]);
Method cleanMethod = cleaner.getClass().getMethod("clean", new Class[0]);
if (cleanMethod != null) {
cleanMethod.invoke(cleaner, new Object[0]);
}
}
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
});
}
The above two methods are awkward. In addition, they can be divided into physical files for recycling calls. This is not very beautiful.