WebRTC QoS方法十三.2（Jitter延时的计算）
一、背景先容一些报文在网络传输中，会存在丢包重传和延时的情况。渲染时需要进行适当缓存，等候丢失被重传的报文或者正在路上传输的报文。
jitter延时计算是确认需要缓存的时间
另外，在检测到帧有重传情况时，也可适当在渲染时间内增加RTT延时时间，等候丢失重传的报文
二、jitter实现原理
JitterDelay由两部分耽误造成：传输大帧引起的耽误和网络噪声引起的耽误。计算公式如下：
https://i-blog.csdnimg.cn/direct/f151e56a77f64e81919505b94794c55a.png
其中：
estimate：信道传输速率的倒数
MaxFrameSize：表示自会话开始以来所收到的最大帧size
AvgFrameSize：表示平均帧巨细，排除keyframe等超大帧
kNoiseStdDevs： 表示噪声系数2.33
var_noise_ms2_： 表示噪声方差
kNoiseStdDevOffset: 表示噪声扣除常数30
实现函数：
JitterEstimator::CalculateEstimate
https://i-blog.csdnimg.cn/direct/8158f24e5c8c47baae52f6361eb373ad.png
1、传输大帧引起的耽误
传输大帧引起的耽误
https://i-blog.csdnimg.cn/direct/85939f9fa4164ff3b659d8c76dad3c2d.png
这个公式的原理是： = *
实现函数：
double FrameDelayVariationKalmanFilter::GetFrameDelayVariationEstimateSizeBased(
double frame_size_variation_bytes) const {
// Unit: * = .
return estimate_ * frame_size_variation_bytes;
} filtered_max_frame_size_bytes
= std::max<double>(kPsi * max_frame_size_bytes_, frame_size.bytes());
constexpr double kPsi = 0.9999;
filtered_avg_frame_size_bytes
是每一帧的加权平均值，但是需要排除key frame这种超大帧
https://i-blog.csdnimg.cn/direct/ef6a225f41664913aeea63d472f88efd.png
estimate_参数计算：
利用一个简化卡尔曼滤波算法，在处理帧耽误变革(frame_delay_variation_ms)的估计，考虑了帧巨细变革(frame_size_variation_bytes)和最大帧巨细(max_frame_size_bytes)作为输入参数。
void FrameDelayVariationKalmanFilter::PredictAndUpdate(
double frame_delay_variation_ms,
double frame_size_variation_bytes,
double max_frame_size_bytes,
double var_noise) {
// Sanity checks.
if (max_frame_size_bytes < 1) {
return;
}
if (var_noise <= 0.0) {
return;
}
// This member function follows the data flow in
// https://en.wikipedia.org/wiki/Kalman_filter#Details.
// 1) Estimate prediction: `x = F*x`.
// For this model, there is no need to explicitly predict the estimate, since
// the state transition matrix is the identity.
// 2) Estimate covariance prediction: `P = F*P*F' + Q`.
// Again, since the state transition matrix is the identity, this update
// is performed by simply adding the process noise covariance.
estimate_cov_ += process_noise_cov_diag_;
estimate_cov_ += process_noise_cov_diag_;
// 3) Innovation: `y = z - H*x`.
// This is the part of the measurement that cannot be explained by the current
// estimate.
double innovation =
frame_delay_variation_ms -
GetFrameDelayVariationEstimateTotal(frame_size_variation_bytes);
// 4) Innovation variance: `s = H*P*H' + r`.
double estim_cov_times_obs;
estim_cov_times_obs =
estimate_cov_ * frame_size_variation_bytes + estimate_cov_;
estim_cov_times_obs =
estimate_cov_ * frame_size_variation_bytes + estimate_cov_;
double observation_noise_stddev =
(300.0 * exp(-fabs(frame_size_variation_bytes) /
(1e0 * max_frame_size_bytes)) +
1) *
sqrt(var_noise);
if (observation_noise_stddev < 1.0) {
observation_noise_stddev = 1.0;
}
// TODO(brandtr): Shouldn't we add observation_noise_stddev^2 here? Otherwise,
// the dimensional analysis fails.
double innovation_var = frame_size_variation_bytes * estim_cov_times_obs +
estim_cov_times_obs + observation_noise_stddev;
if ((innovation_var < 1e-9 && innovation_var >= 0) ||
(innovation_var > -1e-9 && innovation_var <= 0)) {
RTC_DCHECK_NOTREACHED();
return;
}
// 5) Optimal Kalman gain: `K = P*H'/s`.
// How much to trust the model vs. how much to trust the measurement.
double kalman_gain;
kalman_gain = estim_cov_times_obs / innovation_var;
kalman_gain = estim_cov_times_obs / innovation_var;
// 6) Estimate update: `x = x + K*y`.
// Optimally weight the new information in the innovation and add it to the
// old estimate.
estimate_ += kalman_gain * innovation;
estimate_ += kalman_gain * innovation;
// (This clamping is not part of the linear Kalman filter.)
if (estimate_ < kMaxBandwidth) {
estimate_ = kMaxBandwidth;
}
// 7) Estimate covariance update: `P = (I - K*H)*P`
double t00 = estimate_cov_;
double t01 = estimate_cov_;
estimate_cov_ =
(1 - kalman_gain * frame_size_variation_bytes) * t00 -
kalman_gain * estimate_cov_;
estimate_cov_ =
(1 - kalman_gain * frame_size_variation_bytes) * t01 -
kalman_gain * estimate_cov_;
estimate_cov_ = estimate_cov_ * (1 - kalman_gain) -
kalman_gain * frame_size_variation_bytes * t00;
estimate_cov_ = estimate_cov_ * (1 - kalman_gain) -
kalman_gain * frame_size_variation_bytes * t01;
// Covariance matrix, must be positive semi-definite.
RTC_DCHECK(estimate_cov_ + estimate_cov_ >= 0 &&
estimate_cov_ * estimate_cov_ -
estimate_cov_ * estimate_cov_ >=
0 &&
estimate_cov_ >= 0);
} 2、网络噪声引起的耽误
网络噪声引起的耽误
https://i-blog.csdnimg.cn/direct/cb3e39d06496472587fc4dc435169d9f.png
constexpr double kNoiseStdDevs = 2.33; //噪声系数
constexpr double kNoiseStdDevOffset = 30.0;//噪声扣除常数
var_noise_ms2_ //噪声方差
实现函数：
https://i-blog.csdnimg.cn/direct/09dfc942537044dbb091048ecc8afd38.png
噪声方差var_noise_ms2计算
var_noise_ms2 = alpha * var_noise_ms2_ +
(1 - alpha) *(d_dT - avg_noise_ms_) *(d_dT - avg_noise_ms_);
实现函数：JitterEstimator::EstimateRandomJitter
https://i-blog.csdnimg.cn/direct/1b54485820be47b49f576cf783727d01.png
其中：
d_dT = 实际FrameDelay - 评估FrameDelay
在JitterEstimator::UpdateEstimate函数实现
https://i-blog.csdnimg.cn/direct/90f0522e6e2941b58810bed37a4dbb79.png
实际FrameDelay = （两帧之间实际接收gap - 两帧之间实际发送gap）
在InterFrameDelayVariationCalculator::Calculate函数实现
absl::optional<TimeDelta> InterFrameDelayVariationCalculator::Calculate(
uint32_t rtp_timestamp,
Timestamp now) {
int64_t rtp_timestamp_unwrapped = unwrapper_.Unwrap(rtp_timestamp);
if (!prev_wall_clock_) {
prev_wall_clock_ = now;
prev_rtp_timestamp_unwrapped_ = rtp_timestamp_unwrapped;
// Inter-frame delay variation is undefined for a single frame.
// TODO(brandtr): Should this return absl::nullopt instead?
return TimeDelta::Zero();
}
// Account for reordering in jitter variance estimate in the future?
// Note that this also captures incomplete frames which are grabbed for
// decoding after a later frame has been complete, i.e. real packet losses.
uint32_t cropped_prev = static_cast<uint32_t>(prev_rtp_timestamp_unwrapped_);
if (rtp_timestamp_unwrapped < prev_rtp_timestamp_unwrapped_ ||
!IsNewerTimestamp(rtp_timestamp, cropped_prev)) {
return absl::nullopt;
}
// Compute the compensated timestamp difference.
TimeDelta delta_wall = now - *prev_wall_clock_;
int64_t d_rtp_ticks = rtp_timestamp_unwrapped - prev_rtp_timestamp_unwrapped_;
TimeDelta delta_rtp = d_rtp_ticks / k90kHz;
// The inter-frame delay variation is the second order difference between the
// RTP and wall clocks of the two frames, or in other words, the first order
// difference between `delta_rtp` and `delta_wall`.
TimeDelta inter_frame_delay_variation = delta_wall - delta_rtp;
prev_wall_clock_ = now;
prev_rtp_timestamp_unwrapped_ = rtp_timestamp_unwrapped;
return inter_frame_delay_variation;
} 评估FrameDelay = estimate * (FrameSize – PreFrameSize) + estimate
评估FrameDelay实现函数：
double FrameDelayVariationKalmanFilter::GetFrameDelayVariationEstimateTotal(
double frame_size_variation_bytes) const {
double frame_transmission_delay_ms =
GetFrameDelayVariationEstimateSizeBased(frame_size_variation_bytes);
double link_queuing_delay_ms = estimate_;
return frame_transmission_delay_ms + link_queuing_delay_ms;
} 3、jitter延时更新流程
https://i-blog.csdnimg.cn/direct/80c0627093824ae381f557ab369cadbc.png
三、RTT延时计算
VideoStreamBufferController::OnFrameReady函数，在判断帧有重传情况时，还会根据实际情况，在渲染帧时间里面增加RTT值。
https://i-blog.csdnimg.cn/direct/8c75ffddda4f41e3951e9d0adae73ad4.png
JitterEstimator::GetJitterEstimate根据实际配置，可以在渲染时间中适当增加一定比例的RTT延时值。
https://i-blog.csdnimg.cn/direct/57faef7d77dd4e6e9b2fc4184a6f762f.png
四、参考
WebRTC视频接收缓冲区基于KalmanFilter的耽误模型 - 简书在WebRTC的视频处理流水线中，接收端缓冲区JitterBuffer是关键的构成部分：它负责RTP数据包乱序重排和组帧，RTP丢包重传，请求重传关键帧，估算缓冲区耽误等功能...https://i-blog.csdnimg.cn/blog_migrate/003a2ce7eb50c2e24a8c624c260c5930.pnghttps://www.jianshu.com/p/bb34995c549a
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。
页:
[1]