`no such file or directory` when mirroring with wget

Question

I've got a problem with making a correct loop in React. I want to fetch data from JSON to don't repeat components. I tried to make two loops and then two maps, but everything was in bad order. The other problem is that "description" is also an array that's why I'm not able to deal with it

JSON:

{
  "oswiecim": [
    {
      "header": "Oświęcim Zasole",
      "description": [
        "Rejon ulic św Maksymiliana Kolbego",
        "i Stanisławy Leszczyńskiej"
      ]
    },
    {
      "header": "Oświęcim Zasole",
      "description": [
        "Rejon ulic Więźniów Oświęcimia",
        "Obozowej, Polnej i Legionów"
      ]
    },
    {
      "header": "Stare Miasto",
      "description": [
        "Rejon Rynku i Placu ks. Jana Skarbka oraz ",
        "ulic Zamkowej i Władysława Jagiełły"
      ]
    },
    {
      "header": "Stare Miasto",
      "description": [
        "Cmentarz Parafialny oraz rejon",
        "ul. Wysokie Brzegi."
      ]
    },
    {
      "header": "Osiedle Chemików",
      "description": [
        "Największa pod względem liczby ludności",
        "dzielnica Oświęcimia"
      ]
    }
  ]
}

React:

import '../../styles/selection/Selection.scss'
import { useEffect, useState } from 'react';

const Selection = () => {
    const [data, setData] = useState({})
    const getData = async () => {
        await fetch('https://jsoneditoronline.org/#left=cloud.b95a27020e1c45e9b3a7c95a74fc5d49', {
            headers: {
                'Content-Type': 'application/json',
                'Accept': 'application/json'
            }
        })
            .then(res => res.json())
            .then(data => {
                setData(data)

            })
    }
    useEffect(() => {
        getData()
    }, [])

    const headers = []
    const descriptions = []
    for (const item of data.oswiecim) {
        headers.push(item.header)

        descriptions.push(item.description)

    }



    return (
            <div className="selection">
                    {headers.map(item => (
                        <h1>{item}</h1>
                    ))}
                  {descriptions.map(item => (
                    item.map(elem => (
                        <p>{elem}</p>
                    ))
                ))}
            </div>
    );
}

export default Selection;

The result should look like this:

Solution

Your problem originates from the behavior of wget to save the URL http://clojuredocs.org/clojure_core to a file named ./clojuredocs.org/clojure_core, but the URL http://clojuredocs.org/clojure_core/ (notice the trailing slash) to a file named ./clojuredocs.org/clojure_core/index.html.

Once the file ./clojuredocs.org/clojure_core is created, following downloads of e.g. http://clojuredocs.org/clojure_core/something are doomed to fail, because wget can't create a directory ./clojuredocs.org/clojure_core anymore.

This was reported as bug #29647 on the GNU Wget Bugtracker.

With the provided patch (which obviously didn't make into the official source code) this problem vanishes and wget is forced to create the directory first. So, the download can continue.

However, http://clojuredocs.org/clojure_core gets saved as ./clojuredocs.org/clojure_core.1, not as ./clojuredocs.org/clojure_core/index.html.

I cannot judge if the link-converter (-k) is smart enough to make the links in this mirrored local copy working... I stopped the download after a few minutes. (I'm too impatient ;))

Comments

[mpy]

http://clojuredocs.org/clojure_core get's saved as a file named clojure_core. That's why no folder clojure_core can be created. Perhaps you can work around with -nd which doesn't create a directory hierarchy.

[gebby]

Yeah, I see that. It works when I create the folder clojure_core first so that any other file named clojure_core will be renamed to clojure_core1 and so on. Question is, why does it do that? I've tried the -nd option and it just causes other problems, e.g it gets stuck in a loop trying to download robots.txt so many times.

[gebby]

I had no luck with that patch. It introduces undefined methods that break the build. That site has major issues, you knew right canceling the download. It needs a lot of baby sitting and manual intervention. I gave up too after about 20mb. I'm going to have to work with that. Thanks for the answer, I wouldn't have looked in the bugtracker.

[Amila Senadheera]

@Stairss, check this out!

[Stairss]

thank you, this is exactly what I wanted :)

[Stairss]

I really appreciate your response, it helped me a lot :) Can I ask you what's the reason for the question mark right here: {data.oswiecim?.map()}?

[Priyank Kachhela]

That stands for optional chaining. If your api does not return oswiecim in your data then by using this chaining you can prevent page break or error in future. You can find more info about optional chaing here: developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/…