音视频：JavaCV 采集摄像头和麦克风数据推送RTMP流

需要进行简单的音视频编程，如果不是特别熟悉C/C++，那么JavaCV应该是比较好的选择，下面记录一下JavaCV 采集摄像头和麦克风数据推送RTMP流的方法。

同时采集视频和音频需要最好采用不同的线程进行。

1. 视频采集

视频采集使用OpenCVFrameGrabber（当然也可以使用FFmpegFrameGrabber）:

public class VideoRecorder implements Runnable {
 
	private static final int VIDEO_DEVICE_INDEX = 0;
	private FFmpegFrameRecorder recorder;
	private int width, height;
	public VideoRecorder(FFmpegFrameRecorder recorder, int width, int height) {
		this.recorder = recorder;
		this.width = width;
		this.height = height;
	}
 
	@Override
	public void run() {
		try {
			OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(VIDEO_DEVICE_INDEX);
			grabber.setImageWidth(width);
			grabber.setImageHeight(height);
			grabber.start();
			
			long startTS = 0, videoTS = 0;
			Frame frame = null;
			while (!Thread.interrupted() && (frame = grabber.grab()) != null) {
				if (startTS == 0) {
					startTS = System.currentTimeMillis();
				}
				videoTS = 1000 * (System.currentTimeMillis() - startTS);
				if (videoTS > recorder.getTimestamp()) {
					recorder.setTimestamp(videoTS);
				}
				recorder.record(frame);
			}
			
			grabber.close();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

这里的recorder(FFmpegFrameRecorder)是推流使用的，即OpenCVFrameGrabber采集的视频流推送到recorder中，具体在下面的rtmp推流中说明。

2. 音频采集

音频采集直接采用Java的API，读取麦克风的数据：

public class AudioRecoder implements Runnable {
 
	private FFmpegFrameRecorder recorder;
	private int channels;
	private int sampleRate;
	public AudioRecoder(FFmpegFrameRecorder recorder, int sampleRate, int channels) {
		this.recorder = recorder;
		this.sampleRate = sampleRate;
		this.channels = channels;
	}
 
	@Override
	public void run() {
		try {
			AudioFormat format = new AudioFormat(Float.valueOf(sampleRate), 16, channels, true, false);
			TargetDataLine line = (TargetDataLine) AudioSystem.getLine(new DataLine.Info(TargetDataLine.class, format));
			line.open(format);
			line.start();
 
			int sampleRate = (int) format.getSampleRate();
			int numChannels = format.getChannels();
			byte[] buffer = new byte[sampleRate * numChannels];
 
			while (!Thread.interrupted()) {
				int nBytesRead = 0;
				while (nBytesRead == 0) {
					nBytesRead = line.read(buffer, 0, line.available());
				}
				int nSamplesRead = nBytesRead / 2;
				short[] samples = new short[nSamplesRead];
				ByteBuffer.wrap(buffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);
				ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);
				recorder.recordSamples(sampleRate, numChannels, sBuff);
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
 
}

需要注意的是，指定的位数是16，所以需要将采集的byte转化为short（小端模式），同样的，这里的recorder(FFmpegFrameRecorder)是推流使用的。

3. rtmp推流

rtmp推流采用FFmpegFrameRecorder，format设置为FLV，视频编码采用H264，音频编码采用AAC。

public class Recoder {
 
	
	public static void main(String[] args) throws Exception {
		String rtmpURI = "rtmp://127.0.0.1:1935/app/test";
		int width = 1280, height = 720, sampleRate = 44100, channels = 2;
		FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(rtmpURI, width, height, channels);
		// Video
		recorder.setInterleaved(true);
		recorder.setVideoOption("tune", "zerolatency");
		recorder.setVideoOption("preset", "ultrafast");
		recorder.setVideoOption("crf", "28");
		recorder.setVideoBitrate(2000000);
		// H.264
		recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
		recorder.setFormat("flv");
		recorder.setFrameRate(25);
		recorder.setGopSize(25 * 2);
		// Audio
		recorder.setAudioOption("crf", "0");
		recorder.setAudioQuality(0);
		recorder.setAudioBitrate(192000);
		recorder.setSampleRate(sampleRate);
		recorder.setAudioChannels(channels);
		// AAC
		recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
		
		recorder.start();
		Thread vt = new Thread(new VideoRecorder(recorder, width, height));
		Thread at = new Thread(new AudioRecoder(recorder, sampleRate, channels));
		
		vt.start();
		at.start();
		
		vt.join();
		at.join();
		
		recorder.close();
	}
}

这里需要注意的是，启动音视频采集线程后，需要将线程加入到主线程中：

vt.join();
at.join()

4. 效果预览

可以使用VLC拉流查看recorder的推流效果：

注意rtmp服务器需要自己搭建。