dev detects wrong/both version(s) of yarn

[Lythenas]

In a project that contains this in package.json:

  "engines": {
    "yarn": ">=1.7.0 <2",
    "node": ">=16"
  },

and has both .yarnrc and yarn.lock files.

(For reference the project where I tried this is: https://github.com/eclipse-theia/theia)

Using dev I get the following:

env +nodejs.org>=18.3 +yarnpkg.com +classic.yarnpkg.com +git-scm.org

It picks up both +yarnpkg.com and +classic.yarnpkg.com.

I guess mostly by coincidence +yarnpkg.com gets precedence over +classic.yarnpkg.com which means it uses yarn 4.x instead of yarn 1.x.

IMO the correct behavior would be to only detect +classic.yarnpkg.com since package.json specifies yarn version <2.

---

Somewhat related: Is there a way to turn off an automatically detected package from e.g. pkgx.yaml?

[michaelessiet]

If you're not making use of a package you can remove it from the environment using e.g env -yarnpkg.com

[Lythenas]

Yes that works temporarily but there is no way to make it persistent for dev, right?

[michaelessiet]

There is, create a pkgx.yaml file and define the dependencies there like so

dependencies:
  [email protected] node@20 [email protected]

You can use the names or the pkgx display name e.g nodejs.org@20, however I recommend using the pkgx display name in yarns case, just to be specific of which type of yarn

[Lythenas]

Yes, what I'm saying is that this only adds packages. The once that are auto-detected are still there and not removed.

$ cat pkgx.yaml 
dependencies:
  [email protected] node@20 [email protected]
$ dev
env +nodejs.org^20 +yarnpkg.com +classic.yarnpkg.com~1.22 +git-scm.org +python.org~3.10

If I remove the yarn.lock so it does not pick up +yarnpkg.com. However that is part of the repo so deleting it is not really an option.

[Lythenas]

I believe for other dependencies this problem probably does not occur because (if I remember correctly) if there are multiple versions of one package the version constraints are merged in some way. But because yarn 1.x and yarn >2 have different package names (classic.yarnpkg.com and yarnpkg.com), this merging does not happen.

[michaelessiet]

@mxcl Do we have something to handle this better already? Assigned from defining dependencies I haven't come across anything else

[jhheider]

likely the solution is if pkgx.yml.exists() skip_scanning_other_files().

[michaelessiet]

If we add this it will definitely need to go into the docs that if pkgx.yml is defined then you'll need to explicitly define the pkgs needed in your project as the auto detect feature will be turned off as to not clash with the pkgx.yml dependencies

[Lythenas]

likely the solution is if pkgx.yml.exists() skip_scanning_other_files().

I guess that would also be ok. Although maybe you want to autodetect and use additional packages from pkgx.yml.

I think the issue originates here:

pkgx/src/utils/devenv.ts

Lines 76 to 82 in 4787466

	case ".yarnrc":
	pkgs.push({ project: "classic.yarnpkg.com", constraint })
	await read_YAML_FM(path)
	break
	case "yarn.lock":
	pkgs.push({ project: "yarnpkg.com", constraint })
	break

If you have both files you get both packages, since they have different names. I don't have good knowledge of all the yarn versions and I don't know what the reasoning for using two different packages names for yarn is and I don't know what the reasoning is to to choose yarn <2 for .yarnrc but yarn >=2 for yarn.lock.

But for .yarnrc the constraint could also be something like:

pkgs.push({ project: "yarnpkg.com", constraint: new semver.Range("<2") })

If I understand correctly this constraint would then be merged with others and resolve to a single package version.

Although that would only work if you move the versions from classic.yarnpkg.com into yarnpkg.com.

[michaelessiet]

We can't merge the 2 as they are 2 different packages. This is something that has confused many yarn users... even me. And I'm not really sure about the .yarnrc preference for classic.yarnpkg.com. @jhheider do you have more context on this?

[jhheider]

I believe .yarnrc is specific to yarn1.

their migration guide lists Convert your .npmrc and .yarnrc files into .yarnrc.yml. details.

[Lythenas]

But yarn.lock is not specific to yarn>=2, right? So using fixed yarnpkg.com for yarn.lock is not correct, right?

[jhheider]

yes, in some cases, it will not be correct. the implicit assumption was that most yarn users will be using berry; however, it shouldn't likely not push both. if it has already pushed classic.yarnpkg.com it should not also add yarnpkg.com.

[michaelessiet]

@jhheider has the issue of dev pushing yarnpkg.com when classic.yarnpkg.com is already installed been fixed?

[jhheider]

I don't believe I've seen a PR happen for that yet.

[jakobvase]

I'm also hitting this, and another solution is to let pkgx.yml define yarnpkg.com: 'none' or something similar. So there is an escape hatch for when pkgx picks up the wrong packages from the environment.