How to read file starting from a specific number of line within the "while read "?

Question

Recently my project has json requirement, but I've never use json before, so i want to know how to use json effective. I use Volley as my network lib, which support json default, know I can get the JSONObject or JSONArray response, and can get the data by the method such as :

String name = response.optString("name"); 

obviously it's not a good idea, I want to generate java class as my response data, use a model class to wrap the JSONObject or use gson、jackson to generate a java class, anybody can share your experience? thanks.

Solution 1

while IFS= read -r line; do
    # ...
done < <(tail -n "+$lineNumberIs" $dataFile)

tail -n +K (with the plus sign) tells tail to start at the specified line number (see the man page).

The <(...) bit is a process substitution. It lets you specify a command sequence and let bash read from it like a file. It's very handy when you want to avoid the effect of the subshell created in a pipeline.

IFS= read -r is used to read the line exactly s it appears in the file, with no spaces or escape sequences removed.

Solution 2

#!/bin/bash
if [ $# -eq 0 ]; then
        echo "Please execute $0 with linestoskip parameter"
        exit 0
fi
linestoskip=$1
Counter=0
dataFile='/etc/fstab'
while read line
do
        if [ $Counter -ge $linestoskip ]; then
                echo $line
        fi
        Counter=`expr $Counter + 1`
done < $dataFile

This script expects number of lines to skip as a parameter. You can do whatever you like to in the inner if condition.

Comments

[Paul Woitaschek]

Why is that no good idea? JSONObject is already a representation. If you want to convert the JSONObject to a custom model, do so. Just iterate the JSONObject, JSONArray or whatever and create your own datastructure, using normal ArrayLists, HashMaps or whatever you need.

[Ahmed Zain El Dein]

is that the only want there is no syntax for the psuedo code that i put above in my question like while read from .... ? at all

[fiatux]

In bash, you can use arithmetic expressions: ((Counter++)) so you don't have to call out to expr.

[Ahmed Zain El Dein]

thank u :) u mean that it is handy if i want to avoid the waste resources on reading those N unnecessary lines and start directly reading from this specific line number , did i understand u right ?

[fiatux]

Consider this: x=foo; echo bar | read x; echo $x -- Would you be surprised that it outputs "foo"? If you wanted to see "bar" you would write x=foo; read x < <(echo bar); echo $x

[Ahmed Zain El Dein]

yes i understand what u say about x= foo etc .... buut actaully i dont get what is te relation between this and my question :) :)

[fiatux]

The tail command lets you start reading from a specific number. The process substitution is what I called "handy" -- to relate to your question, it's a technique where the output of one command can easily be redirected into another without side effects: the while loop gets the output of tail, so the first read command gets the first line you want. Also, other people will read your question and hopefully learn something from my answer.

[Ahmed Zain El Dein]

ok but what i want to make sure f it is that using this code it will never start reading files from the beginning and skip passing them along to while or whatever and will directly start reading from the specified line number , as if he made index to lines and start from the specified one , am i right or i misunderstand ?

[fiatux]

I don't understand what you are unsure about. Do you mean "what iF tail -n +$lineNumber suddenly stops working properly"? Have you tried it?

[Ahmed Zain El Dein]

it works and i dont doubt in ur answer i am asking abotu something else above in the previous comment and tell u what i understand ,so i understand right or also tail -n +k acts as if it reads the file from the first line and count and skip those lines until it reaches line number K then start to pass them along to while or whatever other command ? that is all :) i am asking about its working mechanism :)

[fiatux]

I don't know how tail is implemented. You can find the source code and check for yourself. I would think that it's well tested and certainly a compiled C program will be faster than an interpreted shell script.

[abe]

I choose jackson at first, but I found it's lib.jar has 6000+ method which let my project nearly reach 65535 problem, so I use gson instead. thanks

[Sergiy Kolodyazhnyy]

Consider editing your answer to mention IFS= read -r line to avoid leading spaces/tabs getting removed.