endianness,byte-order , Why conditional byte order is bad?

Why conditional byte order is bad?


Tag: endianness,byte-order

Linus claims[12] that conditional byte order is worse than silly.

The first thing comes to my mind is ZFS but surely there must be other examples.

He wrote:

The only sane model is to specify one fixed byte order. Seriously. It's equally portable, it generates better code - even on architectures that then have to unconditionally do byte order swapping - and it's simpler to add static type checks for etc. It's literally less code and faster to do a "bswap" instruction than to do a conditional test of some variable (even if you can then avoid the bswap dynamically)

I think conditional byte order may generate longer code but it should be faster on machines that uses same byte order. It seems that most of his views are about aesthetics of code. I am not an expert so, I would like to see a more detailed explanation of his points.


You asked for an explanation of Linus's claims. You are mistaken about them being about aesthetics.

When Linus says "[fixed byte order] generates better code", he is talking mostly about better (i.e. faster) machine code.

When he says "it's simpler to add static type checks for etc." he is probably talking about C source code, but this is not a matter of aesthetics. Simpler code is easier to understand, easier to maintain, and has less surface area for bugs. Often (but not always) it also generates better machine code.

When he says it's "faster to do a 'bswap' instruction than to do a conditional test of some variable" he is again talking about machine code, saying basically that being able to have the data in native order does you no good, performance-wise, if you cannot rely on the data to be in native order. It is better, he claims, to be able to rely on the data to be in the wrong order than to have to test what order it's in before you use it.

Linus argues that all of the above applies even when the protocol's chosen byte order differs from the machine's native byte order. Elsewhere in that conversation, he and others observed that even if the data are not in native byte order, there are many operations you might perform on it that don't care about byte order. Moreover, if you know in advance what the byte order is, then for some other operations that order can be accommodated at compile time in a way that avoids run-time byte swapping.

Linus's argument is compelling.


How can a C++ template be specialized for all 32-bit POD types?

I've developed a simple template function for swapping the byte order of a single field: template <typename T> inline void SwapEndian(T& ptr) { char *bytes = reinterpret_cast<char*>(&ptr); int a = sizeof(T) / 2; while (a--) { char tmp = bytes[a]; int b = sizeof(T) - 1 - a; bytes[a] =...

Blowfish encryption and decryption across c sockets

I'm using the following implementation of blowfish (with some stuff cut out of the middle) to encrypt and decrypt messages. I am attempting to encrypt a message on a big endian machine (SunOS 5.10), and then send it over a socket to a little endian machine (linux). I can encrypt...

Trouble capturing IP packets with libpcap

First the structs: /* Ethernet addresses are 6 bytes */ #define ETHER_ADDR_LEN 6 /* Ethernet header */ struct sniff_ethernet { u_char ether_dhost[ETHER_ADDR_LEN]; /* Destination host address */ u_char ether_shost[ETHER_ADDR_LEN]; /* Source host address */ u_short ether_type; /* IP? ARP? RARP? etc */ }; #define ETHERTYPE_IP 0x0800 /* IP */ /*...

Store typed array in ArrayBuffer

I have an ArrayBuffer of data which is composed of Uint8s. Each is in network byte order (big-endian). My goal is to convert each Uint8 into little-endian order, but then put them back in the ArrayBuffer. I know I can easily separate the individual Uints by using a typed array,...

Why do inet_ntoa and inet_ntop “reverse” the bytes?

This is a rather basic problem with which, to my surprise, I've had a problem today. It looks to me like inet_pton and inet_ntoa are reversing the bytes of the IP address they're given: DWORD IP; inet_pton(AF_INET, "", &IP); printf("%08X\n", IP); This will print 0100A8C0. And well, if we break...

Clean way to make portable endian-correct file-reading / writing code in C++

I want to write some C++ code that can read and write from files in an endian-correct way. More exactly, I want to be able to read a particular type of file, whose endianness I can easily detect (its magic number being reversed or not). But how would I then...

gethostbyname and endianness - how are the bytes returned?

On my (Intel) x86 machine, I've noticed that if I printf the results of gethostbyname for localhost, I get 100007F, even though the MSDN documentation states it should return the IP in network byte order, aka big endian. I searched a bit and found this topic. Based on the answers...

Do word size and endianness interplay when writing cross platform bit level code?

I was just looking at this answer which gives the following sample code to convert an int to an array of bytes: int intValue; byte[] intBytes = BitConverter.GetBytes(intValue); if (BitConverter.IsLittleEndian) Array.Reverse(intBytes); byte[] result = intBytes; I looked up Endianness and found that the reversal of bytes (or lack thereof) is...

Convert a char array into an integer (big and little endian)

I am trying to convert a char array into integer, then I have to increment that integer (both in little and big endian). Example: char ary[6 ] = { 01,02,03,04,05,06}; long int b=0; // 64 bits this char will be stored in memory address 0 1 2 3 4 5...

Java: Efficiently converting an array of longs to an array of bytes

I have an array of longs I want to write to disk. The most efficient disk I/O functions take in byte arrays, for example: FileOutputStream.write(byte[] b, int offset, int length) ...so I want to begin by converting my long[] to byte[] (8 bytes for each long). I'm struggling to find...

Does zlib's “uncompress” preserve the data's original endianness, or does it do an endian conversion?

I am working with legacy C++ code that accesses two-byte integer data compressed in a sqlite database. The code uses zlib's uncompress function to extract the data, which comes out on my little-endian machine as little-endian values. To allow for the possibility that this code may be ported to big-endian...

What is the most significant byte of 160 bit hash for arithmetic operations?

Could somebody help me to understand what is the most significant byte of a 160 bit (SHA-1) hash? I have a C# code which calls the cryptography library to calculate a hash code from a data stream. In the result I get a 20 byte C# array. Then I calculate...

Explanation of HEX value representation and Endianess

I was working on a script to basically output some sample data as a binary blob. I'm a new intern in the software field and vaguely remember the idea of endianness. I realize that the most significant bits for big-endian starts at the top and works down the memory block....

Porting C endianness & pointers black magic to Swift

I'm trying to translate this snippet : ntohs(*(UInt16*)VALUE) / 4.0 and some other ones, looking alike, from C to Swift. Problem is, I have very few knowledge of Swift and I just can't understand what this snippet does... Here's all I know : ntohs swap endianness to host endianness VALUE...

How to byte reverse NSData output in Swift the littleEndian way?

Hey guys I have this output from NSData: <00000100 84000c00 071490fe 4dfbd7e9> So how could I byte reverse it in Swift and have this output: <00000001 0084000c 1407fe90 fb4de9d7>?...

How do I do a bit-wise XOR on NSData in Objective-C?

I have two NSData objects, data1 and data2, and I'd like to do a bit-wise XOR and store the result in a third NSData object, xorData. The first thing I tried was this: *data1.bytes^*data2.bytes; but it gave an error of: Invalid operands to binary expression ('const void' and 'const void')...

Read libcap file with specific endianness

I wrote a c-lang program to read a .pcap file.What fogs me is that the data I read was with a different endianness as to WireShark. I'm working on X86 ach, as I can see, it's LittleEndian. So, can I read the .pcap file with BigEndian? How? Code fragments: /*...

Why conditional byte order is bad?

Linus claims[12] that conditional byte order is worse than silly. The first thing comes to my mind is ZFS but surely there must be other examples. He wrote: The only sane model is to specify one fixed byte order. Seriously. It's equally portable, it generates better code - even on...

How to swap endianness of Int16 without BitConverter

I need to read binary files containing millions of Int16 stored as big endian. My first method was to use BitConverter and Array.Reverse() but that appears too slow for my purpose. Is there a way to do it with bitwise arithmetic instead ?...

Fast way to switch bits order? [closed]

I have a binary like this 10011011 My data store like this 10,01,10,11, but I want to reorder like this 11100110 The data look like 11,10,01,10.This operation same as ByteOrder convert but in bits level. Any fast bitop way to do this ? Currently, I have to decode this to...

How can I simulate an environment where BitConverter.IsLittleEndian is the opposite for my unit tests?

I am using the two methods BitConverter.GetBytes and Array.Reverse to read and write binary data from a file due the endianness. My unit tests are passing and the implementation seems fine. How can I simulate an environment where BitConverter.IsLittleEndian is the opposite for my unit tests?...