var raw = []byte("On modern Linux systems, the in-kernel random number generator in /dev/random is considered cryptographically secure and, crucially, no longer blocks. The previously non-blocking random source /dev/urandom is also updated to use the same CSPRNG (Cryptographically Secure PseudoRandom Number Generator). Even with these changes, you should keep running rngd as it allows the system to combine multiple sources of entropy.\n\nCaveat: randomness for FIPS 140-2 is different and will be covered in a separate post.\n\nHow to access endless high quality random numbers from Linux:\n\nUse /dev/random or /dev/urandom, and run rngd with a Modern Linux Kernel (UEK6 Update 2 or later, or mainline v5.6+)\nUse /dev/random and run rngd on older Linux kernels (e.g. UEK5 or mainline pre v5.6)\nWith the changes introduced to the kernel's random number generator between v4.8 and v5.6, neither /dev/random nor /dev/urandom will block, no matter how much randomness is requested from the system. Documentation has yet to catch up with this brave new world: the manpage for random(4) still talks about /dev/random blocking for an indeterminate length of time. The same CSPRNG algorithm is used for random and urandom: applications can use /dev/random and /dev/urandom interchangeably.\n\nOnly one slight difference was maintained between /dev/random and /dev/urandom: /dev/random retains the ability to block during early system initialization (but not once the system is fully booted). The blocking was retained under the theory that an uninitialized CSPRNG random number generator could provide lower quality random numbers. This might matter for some short lived microservice-type applications that boot their own kernel, but the delay-during-boot is invisible to most applications.\n\nrngd is a userspace daemon that uses multiple entropy sources to constantly refresh the system entropy pool. Even with non-blocking /dev/random, rngd allows the system to use multiple entropy sources instead of relying on just one. By default, the Linux kernel would use only interrupt timing to populate the entropy pool (The kernel can directly use dedicated hardware random number generators as entropy sources, but such devices are uncommon). rngd is the mechanism by which rdrand (the x86 hardware random number instruction) or jitterentropy can be used as a source of entropy. Entropy mixing from rngd can help on virtual machines where interrupt timing may be slightly more deterministic.")
var raw2 = []byte("Most distributions provide and enable rngd by default, and we strongly encourage you to keep rngd running in both virtual and physical machines.\n\nWhen /dev/random used to block\n\nIn the past, we provided guidance to use /dev/random for long lived secrets and /dev/urandom for everything else -- because the cryptographically-secure but blocking behavior of /dev/random would cause many applications to hang or crash. To keep /dev/random cryptographically secure, the kernel would 'debit' the entropy pool for each request to /dev/random and to refuse to provide random numbers when the entropy pool is empty -- breaking applications in the process.\n\nThe daemon rngd was created to keep the entropy pool full by hunting around the system for reliable sources of entropy, keeping the entropy pool full and (hopefully) keeping calls to /dev/random from blocking. Some users switched the random source for their applications from /dev/random to /dev/urandom, though others argued that the PRNG for /dev/urandom was not sufficiently cryptographically secure. In either case, the blocking behavior of /dev/random severely limited the ability of applications and users to depend on it as a reliable source of random numbers.\n\nUnblock with Modern Linux kernels\n\nIn the mainline linux kernel v5.6, a series of eight patches from Andy Lutomirski were submitted:\n\nI believe that Linux's blocking pool has outlived its usefulness. Linux's CRNG generates output that is good enough to use even for key generation. The blocking pool is not stronger in any material way, and keeping it around requires a lot of infrastructure of dubious value.\n\nThat simple statement is the culmination of years of changes to the kernel's random number generator. Applications will notice this first that /dev/random is no longer a bottleneck and second, perhaps, that a read from there is hundreds, perhaps thousands, of times quicker than it used to be.\n\nThe days of dealing with applications hanging, or at least running very slowly, because they can't get to /dev/random are gone. We no longer need to recommend using /dev/urandom to avoid hangs and explaining that, no, contrary what you have read on the Internet, /dev/urandom is not insecure.\n\nBetween mainline Linux kernel versions v4.8 and v5.6 the kernel's CSPRNG was changed to use ChaCha20, the blocking and non-blocking entropy pools were unified and finally, after a few relatively minor changes, removal of the blocking behavior altogether.\n\nThese changes were backported to UEK6u2 (kernel-uek-5.4.17-2102 and higher) so customers running our latest kernels will reap the benefits of faster random numbers.\n\nEven though /dev/random no longer blocks, you need to run rngd! On its own, the Linux kernel usually only add entropy from interrupt timings. Although, since mainline 3.16, hardware RNGs do contribute to the kernel's entropy pool, trusting that there is a valid hardware RNG present instead of running rngd is not recommended.")
var raw3 = []byte("rngd\n\nThough not as fast as modern implementations in the Linux kernels, users who find themselves on older versions of the kernel can still take advantage of faster /dev/random, or at least not blocking, access by running a daemon to ensure kernel's entropy pools stays full and healthy.\n\nrngd is a daemon which will hunt around your system for sources of entropy and use whatever it finds. All you actually need to do is make sure rng-tools is installed and rngd is running. This is the default on OL7 and OL8 but you can check with:\n\nCopy code snippet\nCopied to ClipboardError: Could not CopyCopied to ClipboardError: Could not Copy\n$ systemctl status rngd\n\nIf rngd is not present then you'll need to install it:\n\nCopy code snippet\nCopied to ClipboardError: Could not CopyCopied to ClipboardError: Could not Copy\n$ yum install rng-tools\n\nand make sure it is enabled and running:\n\nCopy code snippet\nCopied to ClipboardError: Could not CopyCopied to ClipboardError: Could not Copy\n$ systemctl enable --now rngd\n\nThere are several sources of entropy that rngd can use, including:\n\njitterentropy --- a generator that makes use of CPU instruction execution time \"jitter\" at the microarchitectural level.\nvirtio_rng -- that provides a pipeline for entropy from a host to a virtual machine. This is presented to the virtual machine as /dev/hwrng, a hardware RNG though, in fact, there may be no hardware backing it. (virtio_rng dates from a time before jitterentropy and rdrand existed, it is obsolete in most cases now.)\nrdrand/rdseed --- the Intel/AMD instructions that provide a source of entropy from special instructions implemented by the CPU. The entropy from these instructions is \"conditioned\" by using the aesni CPU instructions.\nThere are also other entropy gathering daemons, but we only recommend using rngd on Oracle Linux systems.\n\nSurprisingly, perhaps, the kernel does not use rdrand or rdseed as a source of entropy. Instead, these instructions, when available, are used in selected places to enhance other sources of entropy.\n\nIn Part 2 we will go into more depth about sources of entropy.\n\n/dev/urandom\n\nAs briefly mentioned above, the kernel used to have two distinct random number generators. The two RNGs shared code, algorithms and entropy sources, the only difference is that /dev/random drew entropy from a blocking entorpy pool (making it slow and prone to blocking) and /dev/urandom drew entropy from a non-blocking (making it faster and not prone to blocking).\n\nIn those kernels, there was a theoretical weakness for /dev/urandom: if the entropy pool is empty then you may be able to infer future output of the RNG, but this relies on a successful cryptanalysis of SHA-1. Even though SHA-1 is no longer considered sufficient for cryptographic purposes, this degree of cryptanalysis is not believed to be feasible.\n\nIn those older kernels, /dev/random would block because the blocking entropy pool had been depleted. However, changes to the algorithms used in ealier kernels meant that there was no longer any useful distinction between the blocking and non-blocking entropy pools. The change, mentioned briefly above, was to simply have a single entropy pool. The only distinction between /dev/random and /dev/urandom now is that /dev/random might block just after booting. By the time any applcations get around to starting, even ones that start at boot time, /dev/random will no longer be blocking.\n\nIf your applcation uses /dev/urandom and you've always been worried that you're compromising security, you can stop worrying --- it's the same as /dev/random to all intents and purposes. If your application uses /dev/random and you've always been worried that you're compromising performance, you can stop worrying --- it's the same as /dev/urandom to all intents and purposes.\n\nThere is one case where you should specifically choose /dev/random knowing that it might block for a short while. This one case is when you have a microservice that runs in its own virtual machine. This microservice will start very early in the boot sequence, close to the time where /dev/random might block. You should use /dev/random in this case as this is likely to be in the initialisation phase of the microservice and any random numbers used at that point must be of high quality.")
var t = 200
func f1(td *testing.T) {
wr := bytes.NewBuffer(nil)
fm, err := NewWriter(wr, LSB, 8)
if err != nil {
td.Fatal(err)
}
d := [][]byte{raw, raw2, raw3}
for _, v := range d {
for len(v) > t {
fm.Write(v[:t])
fm.Flush()
v = v[t:]
}
fm.Write(v)
fm.Flush()
}
buf := wr.Bytes()
fmt.Println(len(buf))
}
func f2(td *testing.T) {
wr := bytes.NewBuffer(nil)
fm, err := flate.NewWriter(wr, 3)
if err != nil {
panic(err)
}
d := [][]byte{raw, raw2, raw3}
for _, v := range d {
for len(v) > t {
fm.Write(v[:t])
fm.Flush()
v = v[t:]
}
fm.Write(v)
fm.Flush()
}
buf := wr.Bytes()
fmt.Println(len(buf))
}
func TestName(t *testing.T) {
f1(t)
f2(t)
fmt.Println(len(raw) + len(raw2) + len(raw3))
}
