Return to checkduplicates.pl CVS log

Up to [LON-CAPA] / loncom / localize / localize

Heavily optimized version how to search for duplicate keys:
- Read translation file only once and directly count key occurrences
  (inclusion of lexicon hash not needed anymore; Thanks to Stefan Droeschler for the idea)
- More flexible key matching pattern
  (leading white spaces)
- Optimized key matching pattern (quotes)
- Now also print amount of each duplicate key

#!/usr/bin/perl
# The LearningOnline Network with CAPA
# $Id: checkduplicates.pl,v 1.2 2009/04/08 15:10:22 bisitz Exp $

# 07.04.2009 Stefan Bisitz
# Optimization ideas by Stefan Droeschler

use strict;
use warnings;

my $man = "
checkduplicates - Checks if hash keys in translation files occur more than one time. If so, a warning is displayed.

The found keys and corresponding values need to be changed. Otherwise, there is no gurantee which value is taken. This is dangerous, if same keys but different values are used or if one value is changed but the screen still shows the old value which actually comes from the other occurence.


SYNOPSIS:\tcheckduplicates -h 
\t\tcheckduplicates FILE

OPTIONS:
-h\t\tDisplay this help and exit.

";

my $filename; 
die "Use option -h for help.\n" unless exists $ARGV[0];
#analyze options
if ( $ARGV[0] =~ m/^\s*-h/ ) {
	print $man;
	exit();
}else{
	$filename = ($ARGV[0]);
	die "$filename is not a file.\n" unless -f $ARGV[0];
}


# ----------------------------------------------------------------
# Start Analysis
print "checkduplicates is searching for duplicates in $filename...\n";

# Manually read all stored keys from translation file (inlcuding probable duplicates)
# and count key occurrences in a separate hash.
my %counter;
my $line;
open( FH, "<", $filename ) or die "$filename cannot be opened\n";
while ( !eof(FH) ) {
    $line = readline(FH);
    next if $line=~/^\s*#/; # ignore comments
    #$exprNP=~s/^["'](.*)["']$/$1/; # Remove " and ' at beginning and end
    if ($line =~ m/^\s+["'](.*)["']/) { # Find "..." or '...' key
        $counter{$1}++;
    }
}
close(FH);

# Print all keys which occures more than one time
my $dupl = 0; # total counter to count when a key occurred more than one time
foreach my $count_key (keys %counter) {
    my $count_value = $counter{$count_key};
    if ($count_value > 1) {
        print 'Found '.$count_value.' times key: '.$count_key."\n";
        $dupl++;
    }
}

if ($dupl == 0) {
    print "Be happy - No duplicates found.\n";
} else {
    print "--- Found $dupl duplicate(s) in $filename which need to be corrected!\n";
}

# ----------------------------------------------------------------