Uploaded image for project: 'Funtoo Linux'
  1. Funtoo Linux
  2. FL-10108

metatools: bad upstream artifact broke development workflow with merge-kits

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Normal Normal
    • None
    • None
    • Run merge-kits against a remote branch

      During tonight's live streaming of the Funtoo Ruby next development we hit a major metatools blocker when using merge-kits.

      The issue we encountered and triple verified was related to a new Vivaldi deb package. Their upstream HTTP server was failing to HTTP GET requests for this particular package. This broke merge-kits when trying to generate brower-kit.

      In the branch I am using to test Funtoo Ruby next, I even temporarily disabled this particular artifact from downloading in Vivaldi's Python autogen. But merge-kits fails to pick up that change.

      This leads me to believe some type of local caching after subsequent merge-kits runs is preventing these changes from propagating locally.

      This is a request to make merge-kits more robust when upstream artifacts fail and provide some option or methodology to Funtoo developers to do specific cache wipes to "reset" a certain kit generation when running merge-kits based on a remote branch.

      Right now I am blocked on Funtoo Ruby next development because of this.

      [ERROR 23:37:55] Download failure for https://repo.vivaldi.com/snapshot/deb/pool/main/vivaldi-snapshot_5.4.2718.3-1_armhf.deb: peer closed connection without sending complete message body (received 32505856 bytes, expected 81362464)
      [ERROR 23:37:55] Fetch error: (<metatools.fastpull.spider.FetchRequest object at 0x7f1cf9d414f0>, 'RemoteProtocolError: peer closed connection without sending complete message body (received 32505856 bytes, expected 81362464)')
      [ERROR 23:37:55] Artifact for url https://repo.vivaldi.com/snapshot/deb/pool/main/vivaldi-snapshot_5.4.2718.3-1_armhf.deb referenced in www-client/vivaldi-snapshot could not be fetched.
          Traceback (most recent call last):
            File "/home/ss/Repos/funtoo-metatools/funtoo/pkgtools/ebuild.py", line 361, in wrapper
              await self.generate()
            File "/home/ss/Repos/funtoo-metatools/funtoo/pkgtools/ebuild.py", line 481, in generate
              await self.setup()
            File "/home/ss/Repos/funtoo-metatools/funtoo/pkgtools/ebuild.py", line 347, in setup
              raise BreezyError(f"Artifact for url {artifact.url} referenced in {artifact.catpkgs} could not be fetched.")
          dyne.org.funtoo.metatools.pkgtools.ebuild.BreezyError: Artifact for url https://repo.vivaldi.com/snapshot/deb/pool/main/vivaldi-snapshot_5.4.2718.3-1_armhf.deb referenced in www-client/vivaldi-snapshot could not be fetched.
      [ERROR 23:38:03] An error was encountered when processing www-client/vivaldi-snapshot-5.4.2718.3_p1
      [ERROR 23:38:03] Command failure from merge-kits: cd /home/ss/repo_tmp/source-trees/kit-fixups/browser-kit/curated && doit --fast --fastpull_scope=local
      Traceback (most recent call last):
        File "/home/ss/Repos/funtoo-metatools/bin/merge-kits", line 92, in <module>
          hub.LOOP.run_until_complete(main_thread(args))
        File "/usr/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
          return future.result()
        File "/home/ss/Repos/funtoo-metatools/bin/merge-kits", line 57, in main_thread
          await controller.generate()
        File "/home/ss/Repos/funtoo-metatools/metatools/kit.py", line 892, in generate
          await self.process_all_kits_in_release(method="generate")
        File "/home/ss/Repos/funtoo-metatools/metatools/kit.py", line 868, in process_all_kits_in_release
          await other_pool.run()
        File "/home/ss/Repos/funtoo-metatools/metatools/kit.py", line 754, in run
          await method()
        File "/home/ss/Repos/funtoo-metatools/metatools/kit.py", line 547, in generate
          await self.generate_autogenerated()
        File "/home/ss/Repos/funtoo-metatools/metatools/kit.py", line 496, in generate_autogenerated
          await self.run(self.autogen_and_copy_from_kit_fixups())
        File "/home/ss/Repos/funtoo-metatools/metatools/kit.py", line 156, in run
          await step.run(self)
        File "/home/ss/Repos/funtoo-metatools/metatools/steps.py", line 413, in run
          await self.srctree.autogen(src_offset=self.ebuildloc, scope=self.scope)
        File "/home/ss/Repos/funtoo-metatools/metatools/tree.py", line 109, in autogen
          raise GitTreeError(f"failed autogen in {self.root} -- offset {src_offset}.")
      metatools.tree.GitTreeError: failed autogen in /home/ss/repo_tmp/source-trees/kit-fixups -- offset browser-kit/curated.

      For reference the merge-kits command used to generate that error:

      time merge-kits next --fixups_url https://code.funtoo.org/bitbucket/scm/~siris/kit-fixups.git --fixups_branch "FL-9695/rubies-next-gen-v1"

      The version of metatools being used: master HEAD commit:

      1d16704 (HEAD -> master, origin/master, origin/HEAD) FL-9647: minor massage of code.

            Unassigned Unassigned
            siris siris
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: