BLEU的PHP实现
编辑
500
2021-04-20
参考:http://www.nltk.org/_modules/nltk/align/bleu_score.html,https://blog.csdn.net/Allocator/article/details/79657792,https://blog.csdn.net/guolindonggld/article/details/56966200
参考NLTK和BLEU的公式实现的php版本BLEU计算,目前能够计算一个candidate对应一个reference的BLEU。(虽然通常会有多个reference)
贴下代码
class BLEU
{
public function bleu_score($candidate, $reference)
{
// 计算BP惩罚因子
$bp = $this->brevity_penalty($candidate, $reference);
// 计算修正的n元精确率(Modified n-gram Precision)
$mp = $this->modified_precision($candidate, $reference, 3);
return $bp*exp($mp);
}
/**
* 计算BP
* @param $candidate
* @param $reference
* @return float
*/
private function brevity_penalty($candidate, $reference): float
{
$candidate_ = explode(" ", $candidate);
$reference_ = explode(" ", $reference);
$can_length = sizeof($candidate_);
$ref_length = sizeof($reference_);
if ($can_length >= $ref_length) {
return 1.0;
} else {
// BP = e^(1-r/c)
return exp((1 - $ref_length / $can_length));
}
}
/**
* 计算MP
* @param $candidate
* @param $reference
* @param $n
* @return float|int
*/
private function modified_precision($candidate, $reference, $n = 3)
{
$precisions = [];
for ($i=1; $i<=$n; $i++) {
// 分别计算每个ngram的precision
$candidate_grams = $this->ngram($candidate, $i);
$reference_grams = $this->ngram($reference, $i);
$can_gram_count = 0;
foreach ($candidate_grams as $gram) {
$can_gram_count += substr_count($candidate, $gram);
}
$ref_gram_count = 0;
foreach ($reference_grams as $gram) {
$ref_gram_count += substr_count($reference, $gram);
}
$p = $ref_gram_count / $can_gram_count;
array_push($precisions, $p);
}
return $this->log_sum($precisions) / sizeof($precisions);
}
/**
* 计算数组中每个数log后的加和
* @param array $precisions
* @return float|int
*/
private function log_sum(array $precisions)
{
$sum = 0;
foreach ($precisions as $precision) {
$sum += log($precision);
}
return $sum;
}
/**
* 将一个句子分解成ngram
* @param $sentence
* @param $n
* @return array
*/
private function ngram($sentence, $n)
{
$sentence_ = explode(" ", trim($sentence));
$ngrams = [];
foreach ($sentence_ as $idx => $gram) {
if ($idx <= sizeof($sentence_) - $n) {
$ngram = "";
for ($i = $idx; $i<$idx+$n; $i++) {
$ngram .= " " .$sentence_[$i];
}
$ngram = ltrim($ngram);
array_push($ngrams, $ngram);
}
}
return $ngrams;
}
}
食用方法
(new BLEU())→bleu_score($candidate, $reference)
- 0
- 0
-
赞助
微信赞赏码 -
分享