Perl: Multiply-Shift Hash

I've been using the multiply-shift hash lately and have had good results. It's simple, fast, and well tested.

// Multiply-Shift Hash
static uint64_t hash_msh(uint64_t x) {
    uint64_t prime = 0x9e3779b97f4a7c15; // A large prime constant
    x ^= (x >> 30);
    x *= prime;
    x ^= (x >> 27);
    x *= prime;
    x ^= (x >> 31);
    return x;
}

Whenever I find interesting code like this I like to try and implement it in Perl:

sub hash_msh {
    my $x     = $_[0];
    my $prime = 11400714819323198485;

    $x ^= ($x >> 30);
    $x = multiply_uv($x, $prime);
    $x ^= ($x >> 27);
    $x = multiply_uv($x, $prime);
    $x ^= ($x >> 31);

    return $x;
}

# Perl converts numbers larger than 2^64 - 1 to floating point so
# we 'use integer' to force integer math which retains any overflow.
sub multiply_uv {
    my ($one, $two) = @_;

    use integer;
    my $ret = $one * $two;
    no integer;

    # Convert signed IV to unsinged UV
    if ($ret < 0) {
        $ret += 18446744073709551615;
        $ret += 1;
    }

    return $ret;
}
Leave A Reply

Perl: Calculate number of bits needed to store a number

Quick Perl function to calculate the number of bits required to store a given number.

sub bits_required {
    my ($n) = @_;

    # Handle special case for 0
    return 0 if $n == 0;

    # Use logarithm to calculate the number of bits
    my $bits = int(log($n) / log(2)) + 1;
    return $bits;
}
Leave A Reply

Implmenting PRNGs in Perl

I'm a big fan of the PCG32 PRNG. It's simple, fast, and well documented. As is usually the case when I find interesting code I try and find a way to implement it in Perl. PCG32 poses an interesting problem when one tries to implement it in Perl because all the math is performed using unsigned integers and overflow. Most PRNGs use large numbers and overflow to work, that's their secret sauce.

Perl does not have a native unsigned type so I had to learn the guts of how Perl does math so I could emulate it. I ended up coming up with two different implementations, a native Perl implementation, and a version that uses Math::Int64. Surprisingly the native version was significantly faster.

Both implementations with detailed comments are available in pcg32.pl if you're interested in learning more about how to do integer style math in Perl.

As a bonus I also ported the xoshiro256 family into Perl. Splitmix64 ended up being a little more complicated to port, but I got it as well.

Leave A Reply

C/C++: List all #define macros

I need to target a specific platform with some code I'm writing. You can list all of the #define macros on a given platform with these commands:

gcc -dM -E - < /dev/null
clang -dM -E - < /dev/null

In this big list of macros you should find something to help you target a given platform.

Leave A Reply

FFMPEG: Limit the output to only the encoding stats for quieter scripting

FFMPEG is a great tool, but it is very chatty. If you want to encode a file you'll get about 40 lines of headers and track information, most of which you probably will not read. For scripting purposes you probably only want to see the statistics. If you start FFMPEG, hide the banner, and set the loglevel low you can output only the stats you need

ffmpeg -y -hide_banner -loglevel quiet -stats ...
Leave A Reply

Perl: Using sha256 to hash strings into integers

I need 64bit integers to seed some PRNG work I'm doing. I found out that you can easily create four 64bit numbers from any string by using SHA256.

use Digest::SHA;

my $input = $ARGV[0] || int(rand() * 1000000);

# Get raw bytes and convert to an array of uint64_t
my $hash = Digest::SHA::sha256($input);
my @nums = unpack("Q*", $hash);

print "SHA256 Hashing '$input' to: " . join(", ", @nums) . "\n";

Note: We are using sha256 here, not sha256_hex because we need raw bytes.

Leave A Reply

Komihash is pretty amazing and simple hashing algorithm

I need a simple way to hash strings into 64bit integers so I started some research. Initially I was focusing on xxHash but it's slightly harder to implement than other newer hashes. I ended up landing on the amazing Komihash which bills itself as: "a very fast 64-bit hash function, mainly designed for hash-table, hash-map, and bloom-filter uses".

Komihash is available as a single Komihash.h file and is extremely easy to implement into existing code.

#include <stdio.h>
#include <stdlib.h>
#include "komihash.h"

int main(int argc, char *argv[]) {
    int seed        = 0;
    const char *buf = "Hello world";

    uint64_t hash_num = komihash(buf, strlen(buf), seed);

    printf("Komihash: %s = %llu\n", buf, hash_num); // 3745467240760726046
}

As a bonus it also comes with a PRNG which is equally simple to implement:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include "komihash.h"

int main(int argc, char *argv[]) {
    uint64_t seed1 = time(NULL) * 0x61c8864680b583ebull; // hash64
    uint64_t seed2 = seed1      * 0x61c8864680b583ebull; // hash64

    for (int i = 0; i < 5; i++) {
        uint64_t rand64 = komirand(&seed1, &seed2);
        printf("Komirand: %llu\n", rand64);
    }
}

In fact, I liked it so much I ported it to Perl and called it Crypt::Komihash.

Leave A Reply

Perl: Warning about non-portable numbers

If you get a Perl warning about non-portable numbers like this:

Hexadecimal number > 0xffffffff non-portable at /tmp/x.pl line 19.

It means that you are using a hex number greater than 2^32. I dunno why this is a warning when everything is 64bit these days, but it is. There have been a couple of times I needed this in my code so you can disable these warnings with:

no warnings 'portable';
Leave A Reply

Books of 2025

List of books I read in 2025. Also see the list of 2024. The date indicated denotes the date I started reading the book.

2025-01-06 - Mistborn by Brandon Sanderson - 643 pages
2025-01-17 - Ender's Game by Orson Scott Card - 324 pages
2025-01-22 - Thinner by Stephen King - 318 pages
2025-01-26 - The Dead Zone by Stephen King - 426 pages
2025-02-02 - The Neverending Story by Michael Ende - 377 pages
2025-02-08 - The Well Of Ascension by Brandon Sanderson - 785 pages

Leave A Reply

C: Nanoseconds since Unix epoch

I needed a function to use as a simple seed for srand(). Unixtime in nanoseconds changes very frequently and serves as a semi-decent random seed.

#include <time.h> // for clock_gettime()

// Nanoseconds since Unix epoch
uint64_t nanos() {
    struct timespec ts;

    // int8_t ok = clock_gettime(CLOCK_MONOTONIC, &ts); // Uptime
    int8_t ok = clock_gettime(CLOCK_REALTIME, &ts);  // Since epoch

    if (ok != 0) {
        return 0; // Return 0 on failure (you can handle this differently)
    }

    // Calculate nanoseconds
    uint64_t ret = (uint64_t)ts.tv_sec * 1000000000ULL + (uint64_t)ts.tv_nsec;

    return ret;
}

See also: Microseconds since epoch in Perl

Leave A Reply

Git: Show all changes to a given file

If you want to view the history of all the changes you've made to a specific file in Git you can use the log command in Git. This will show you all the commits that touched that file, and the associated commit message.

git log README.md

If you want to see a diff/patch for each commit you can run:

git log --follow --patch README

This can be very useful to track down when a specific piece of code went into (or was taken out of) a file. Be careful as this command only shows you the diff/patch for the file you request. If the commit contains changes to other files you will not see those unless you view the full commit.

Leave A Reply

Perl: Get microseconds since Unix epoch

I need to get system uptime with a higher resolution that just one second. This is the function I came up with to return uptime in microseconds.

sub micros {
    require Time::HiRes;

    my @p   = Time::HiRes::gettimeofday();
    my $ret = ($p[0] * 1000000) + $p[1];

    return $ret;
}
Leave A Reply

How big is 64bits

Older CPUs are 32bits and could address 4.2 gigabytes of memory. New CPUs are 64bits and can address 18.4 exabytes of memory.

32bits = 4,294,967,296
64bits = 18,446,744,073,709,551,616

Leave A Reply

PHP: Run a command and capture STDOUT and STDERR separately

I need to run a shell command in PHP and capture STDOUT and STDERR separately. This function lets you run a command and returns a hash with the various components and outputs.

function run_cmd($cmd) {
    $start   = hrtime(true);
    $cmd     = escapeshellcmd($cmd);
    $process = proc_open($cmd, [
        1 => ['pipe', 'w'], // STDOUT
        2 => ['pipe', 'w'], // STDERR
    ], $pipes);

    if (!is_resource($process)) { return []; }

    $stdout = stream_get_contents($pipes[1]);
    fclose($pipes[1]);
    $stderr = stream_get_contents($pipes[2]);
    fclose($pipes[2]);
    $exit   = proc_close($process);

    $ret = [
        'exit_code' => $exit,
        'stdout'    => trim($stdout),
        'stderr'    => trim($stderr),
        'cmd'       => $cmd,
        'exec_ms'   => (hrtime(true) - $start) / 1000000,
    ];

    return $ret;
}
Leave A Reply

Linux: Disable DNS cache in systemd

New Linux distributions are enabling local DNS caching. For most users this is a sane default, but as a system administrator this can sometimes get in the way. You can check if your local cache is running and see the statistics with:

resolvectl statistics

Which will tell you how active your cache is, what the hit/miss ratio is, and information about DNSSEC. If you need to clear your local cache you can run:

resolvectl flush-caches

Doing this repeatedly can get cumbersome if you are testing remote DNS frequently. If you want to disable local caching all together you can run:

systemctl disable systemd-resolved.service --now

Remember to update your /etc/resolv.conf to point at the correct DNS server after you're done.

Leave A Reply