Convert a list of decimal values in a text file into hex format

Question

I have a need to convert a list of decimal values in a text file into hex format, so for example test.txt might contain:

131072
196608
262144
327680
393216
...

the output should be list of hex values (hex 8 digit, with leading zeroes):

00020000
00030000
00040000
...

the output is printed into the text file. How to make this with python or linux shell script?

EDIT #1

I missed one extra operation: I need to add 80000000 hex to each of the created hex values. (arithmetic addition, to apply to already created list of hex values).

Solution 1

You can do this using printf and bash:

printf '%08x\n' $(< test.txt)

Or using printf and bc...just...because?

printf '%08s\n' $(bc <<<"obase=16; $(< test.txt)")

---

In order to print the output to a text file just use the shell redirect > like:

printf '%08x\n' $(< test.txt) > output.txt

Solution 2

Three possible solutions (assuming each line is a set of only digits):

For shells like ksh, bash, zsh:

printf '%08x\n' $(<infile)

Only for bash

<file.txt mapfile -t arr;   printf '%08x\n' "${arr[@]}" 

For simpler shells: dash (the default sh in Debian based systems), ash (busybox emulated shell), yash and some default shells in AIX and Solaris:

printf '%08x\n' $(cat infile)

In fact, for a shell like the heirloom version (Bourne like) it needs to be written like (which do work on all posix shells listed above but I strongly recommend to do not use it ):

$ printf '%08x\n' `cat infile`

Answer to EDIT #1

Understand that an hex value of 80000000 will cause overflow on a 32 bit computer (not common this days, but possible). Check that echo "$((0x80000000))" do not print a negative value.

$ for i in $(<infile); do printf '%08x\n' "$(($i+0x80000000))"; done
80020000
80030000
80040000
80050000
80060000

Solution 3

Using awk:

$ awk '/[0-9]/ { printf("%08x\n", $0) }' file
00020000
00030000
00040000
00050000
00060000

Solution 4

Yet another one-liner. Still not python, but which allows comments, empty lines and whatever in the file, and only prints out the result for lines which contain only a number.

perl -ne '/^(\d+)$/ && printf "%08x\n", $1'  $your_file

Given a file like this:

$ cat $your_file
# some numbers,
131072
196608

262144
327680
393216

and empty lines
and whatever...

It prints

00020000
00030000
00040000
00050000
00060000

Solution 5

Here are a couple of one line solutions in PHP:

<?php
foreach (file('test.txt') as $s) echo dechex(+$s), "\n";

<?php
array_map(fn($s) => print dechex(+$s) . "\n", file('test.txt'));

View more solutions

Comments

[minto]

This code add extra 8-bit at the end 00000000.

[Kusalananda]

@minto Did you run the command on a file that had ... on the last line? If so, then yes. It would convert all values to hexadecimal, even empty lines. Data that can not be converted to an integer would be taken as zero.

[minto]

yes, the file have an empty newline at end of file, but empty, no dots.

[Kusalananda]

@minto See updated answer. The updated answer outputs only if the input line contains a number. How big is your file?

[minto]

File have just 64 lines.

[Dani_l]

You should use bc as primary (only?) answer. bc is arbitrary precision, printf will fail for large numbers.

[jesse_b]

@Dani_l: I don't know of any way to reliably pad the bc output with zeros the way OP wants without using printf.

[minto]

How to add 80000000 hex value to each of the created hex values? I mean something like printf "%X\n" $((80000000 + 0x$b)) b=list of hex values

[Grump]

@Dani_l, the OP's requirement to output in 8 hex digit format will fail for large numbers. I assume he meant 8 hexit when he used 8 bit.

[mivk]

@minto : If I understand what you mean, you could replace $1 in the example above with $1+hex("80000000") or $1+2147483648. The output would then be 80020000, 80030000, etc.

[minto]

but I have no perl.

[mivk]

You used linux in your tags. I know of no Linux (or Unix) distribution which doesn't come with Perl in the base install. Even Mac OS X has it.

[Dani_l]

@Grump the OP's requirement is satisfied automatically by the addition operation, as the ouput would always b 8 or more digits, and reducing the number of digits doesn't make sense in an addition operation.

[Grump]

exactly, the requirement is 8 digits, not 8 or more.

[Dani_l]

Probably should use while read instead of for loop of file contents. unix.stackexchange.com/questions/169716/… ; mywiki.wooledge.org/DontReadLinesWithFor

[done]

As I said in the first line: "assuming each line is a set of only digits" it makes no difference to add the complexity of read. @Dani_I

[minto]

@Isaac maybe some clarification should be done regarding hex value of 80000000 that may cause potential overflow on a 32 bit computer. 80000000 is the base address of the flash chip in the embedded system. Base address chosen by device manufacturer, so likely allowed by the processor’s memory controller. I'm not sure it can cause overflow in this case.

[done]

@minto In answer to your comment I'm not sure it can cause overflow in this case I have to say: I do not know if you are running the shell inside the same system that use that memory address. You may be developing and compiling on one computer and then sending the compiled code to the embedded system. Or ... you may be running the shell inside the embedded system, in which case it seems reasonable to expect that the shell has been compiled to run in that system and that the shell has been compiled with the same bit size. But I can not know that both facts are true.