soname: libobs.so.30 vs libobs.so.0

[umlaeute]

Operating System Info

Debian

OBS Studio Version

32.0.4

Expected Behavior

I expect libobs to provide a stable API/ABI, and use proper soversioning for API/ABI breaks.

For the Debian-official packages of obs-studio I would like to be able to express proper versioned dependencies, as this is the only way to get smooth distribution upgrades.

Current Behavior

OBS version	libobs	symlink	dev library
32.0.4	libobs.so.30	libobs.so.0 -> libobs.so.30	libobs.so -> libobs.so.30
30.2.3	libobs.so	libobs.so.0 -> libobs.so.30	libobs.so -> libobs.so.0

resp.:

$ find obs-studio-3* -name "libobs.so*" -ls
   276127   1160 -rw-r--r--   1 user user  1185280 Dec 10  2023 obs-studio-30.0.2/usr/local/lib/libobs.so.30
   276126      0 lrwxrwxrwx   1 user user       12 Dec 10  2023 obs-studio-30.0.2/usr/local/lib/libobs.so.0 -> libobs.so.30
   276125      0 lrwxrwxrwx   1 user user       11 Dec 10  2023 obs-studio-30.0.2/usr/local/lib/libobs.so -> libobs.so.0
   274054   1132 -rw-r--r--   1 user user  1155528 Dec 13 21:59 obs-studio-32.0.4/usr/local/lib/x86_64-linux-gnu/libobs.so.30
   274053      0 lrwxrwxrwx   1 user user       12 Dec 13 21:59 obs-studio-32.0.4/usr/local/lib/x86_64-linux-gnu/libobs.so.0 -> libobs.so.30
   274052      0 lrwxrwxrwx   1 user user       12 Dec 13 21:59 obs-studio-32.0.4/usr/local/lib/x86_64-linux-gnu/libobs.so -> libobs.so.30

a 3rd-party plugin (I tested with obs-color-monitor, but I'm sure the same applies to others as well), will behave slightly different depending on whether it was linked against obs-studio-32.0.4 or obs-studio-30.2.3:

compiled against 32.0.4, I get:

$ readelf -d color-monitor.so  | grep NEEDED | grep libobs
 0x0000000000000001 (NEEDED)             Shared library: [libobs-frontend-api.so.30]
 0x0000000000000001 (NEEDED)             Shared library: [libobs.so.30]

whereas compiled against 30.2.3, I get

$ readelf -d color-monitor.so  | grep NEEDED | grep libobs
 0x0000000000000001 (NEEDED)             Shared library: [libobs-frontend-api.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libobs.so.0]

what is the reasoning for this weird behaviour?

now, because libobs.so.0 is a symlink to libobs.so.30 (in both cases), it might appear that libobs from 32.0.4 is a drop-in-replacement for libobs from 30.2.3.
only... it is not. symbols have been removed from libobs between 30.2.3 and 32.0.4 (e.g. obs_frontend_get_app_config).
the net outcome of this is, that if I compile obs-color-monitor against OBS-30.2.3, I get a binary that is not usable with OBS-32.0.4.

Anything else we should know?

now the problem of incompatible libraries is not anything new, so people came up with solutions for this decades ago: soname versioning.

would it be possible to provide a clear and consistent soname scheme for the libobs* libraries, so it's easier to detect incompatibilities and get inter-package dependencies right?

that is:

• drop the libobs.so.0 symlink; it doesn't serve any good.
• bump the soversion whenever there's a breaking change in the ABI (usually this means: whenever a symbol is removed from the library)
• maybe add a Provides: libobs<soname> (e.g. Provides: libobs30) to your Debian/Ubuntu package, so plugins that want to provide their own debs can add proper dependencies.

[mihawk90]

There was a PR merged for this for 32(?), but it was ultimately reverted after some discussion to be looked at again at a later point. There are some issues especially with the plugin ecosystem on Linux in general that need to be taken care of first.

For reference:

Release Candidate 1 Changes

• Reverted "Removed ability on Linux to load plugins built for older major versions of OBS Studio [tytan652]" [RytoEX]

Beta 2 Changes

• Removed ability on Linux to load plugins built for older major versions of OBS Studio [tytan652]
  • OBS Studio 32 (and future 32.x versions) on Linux can only load plugins linked against libobs 32.

[kkartaltepe]

I expect libobs to provide a stable API/ABI

Unfortunately your expectation does not match reality.

that is:

drop the libobs.so.0 symlink; it doesn't serve any good.
bump the soversion whenever there's a breaking change in the ABI (usually this means: whenever a symbol is removed from the library)
maybe add a Provides: libobs (e.g. Provides: libobs30) to your Debian/Ubuntu package, so plugins that want to provide their own debs can add proper dependencies.

We wont be providing fine grained soversions, so you will need to rebuild things with major updates if the soversion changes land again as they were proposed. The same as every other unstable abi library.

[tytan652]

A pretty bad assumption was made by downstream, our policy about API/ABI was not properly put in place since "forever".

[umlaeute]

thanks for all your answers.

so:

@kkartaltepe wrote:

you will need to rebuild things with major updates if the soversion changes land again as they were proposed. The same as every other unstable abi library.

the point of soversion bumps is to (also) be able to automatically detect whena rebuild is required.

@tytan652 wrote:

A pretty bad assumption was made by downstream, our policy about API/ABI was not properly put in place since "forever".

I think that if a library announces a soname other than 0, then this does communicate that the project cares for API/ABI.
(so rather than "a pretty bad assumption" (by downstream), I would call it "a pretty bad announcement" (by upstream); obviously I'm on the receiving side of the problem, so forgive me my snarkiness)

I can happily live with an unstable API/ABI, if this is clearly communicated.
the best automatic way to communicate this is to set soname=0 and be done with it.

would you consider this?

afaict, on the obs-studio side this means:

• set the soname to 0 in

  obs-studio/cmake/linux/helpers.cmake

  Line 22 in acd4c97

  set(OBS_SOVERSION 30)

  and friends
• drop the symlinks

on the plugin-developers side, nothing really changes: they have to rebuild their plugins anyhow for each new release of obs-studio.

[kkartaltepe]

I think that if a library announces a soname other than 0, then this does communicate that the project cares for API/ABI.
(so rather than "a pretty bad assumption" (by downstream), I would call it "a pretty bad announcement" (by upstream); obviously I'm on the receiving side of the problem, so forgive me my snarkiness)

You can follow x264 packaging which uses the same scheme that was proposed for obs, instead of whatever you are thinking of doing if we announce soversion 0. Except that we actually do intend to avoid making api changes on minor versions.

Again assuming the proposed soversioning proposal lands again.

[umlaeute]

I'm not sure i can follow, esp. i have problems parsing this:

You can follow x264 packaging [...], instead of whatever you are thinking of doing if we announce soversion 0.

if with "x264 packaging" you mean, the Debian packages, then I see that they are currently at soname 165 - which basically means that they have a not-very-stable ABI and therefore do frequent soname bumps. this is what I was suggesting at the top of this issue (and which I have the feeling was shot down).

maybe this is just a misunderstanding: when you said (#13153 (comment)) "The same as every other unstable abi library." and I responded with (#13153 (comment)) "the point of soversion bumps is to (also) be able to automatically detect when a rebuild is required." then I was agreeing with you (as in: yes I know that I have to rebuild for every soname bump; this is what I want as my tools will help me; if however the soversion is bumped arbitrarily (esp. not bumping it when the ABI breaks), this is much harder to work with)

Again assuming the proposed soversioning proposal lands again.

does this mean that the you do plan to have proper soversioning (as in: bump the soversion whenever the ABI breaks)?
any ETA?

[umlaeute]

maybe this conversation started to go downhill because i opened it with

I expect libobs to provide a stable API/ABI

I understand that you cannot/will not make any promises about the long-term stability.

anyhow, what I really wanted to raise with this issue is the 2nd half of my sentence:

I expect libobs to [...] use proper soversioning for API/ABI breaks.

this entire ticket is really about the use of sonames, and not about having a stable ABI.

---

my conclusion in #13153 (comment) was that if you do not want to bump the soname whenever the ABI breaks, then pretty please do use a soname 0 to clearly communicate this.

[kkartaltepe]

my conclusion in #13153 (comment) was that if you do not want to bump the soname whenever the ABI breaks, then pretty please do use a soname 0 to clearly communicate this.

It is unlikely the soversion will change back to 0, for now it is frozen at 30 until/unless another PR to a new versioning scheme lands.

[PatTheMav]

maybe this conversation started to go downhill because i opened it with

I expect libobs to provide a stable API/ABI

I understand that you cannot/will not make any promises about the long-term stability.

anyhow, what I really wanted to raise with this issue is the 2nd half of my sentence:

I expect libobs to [...] use proper soversioning for API/ABI breaks.

this entire ticket is really about the use of sonames, and not about having a stable ABI.

my conclusion in #13153 (comment) was that if you do not want to bump the soname whenever the ABI breaks, then pretty please do use a soname 0 to clearly communicate this.

FWIW I don't think that you're wrong or that your assumptions were wrong. And at least IMO it is the project that is playing fast and loose with platform conventions here and we have to get "this house in order".

I naively tried to "fix" this by encoding the informal choice of the project to only introduce breaking changes with every major release into the actual library file name (while temporarily keeping a symlink to a version "0" around for legacy plugins) because it felt deeply "wrong" to me for any plugin to potentially crash the application if it looks at any arbitrary libobs function funny.

One counter was that this is unnecessarily harsh because many plugins might not even use the functions whose ABI has changed and thus might merrily continue to run fine even in the new major version and thus introduce a whole lot of of unnecessary friction.

My retort probably boiled down to this being nothing less than a whole lot of "happy accidents" and that using library versioning would ensure that any plugin running in OBS is compatible with the version of libobs contained within it and not following this practice turns every existing plugin into an aforementioned "time bomb".

What won the argument for the time being (again, IIRC) was that we should not publicly announce such a change in the same version that we introduce it, which is very much a valid point.

So the symlink to version "0" was kept to avoid breaking existing "time bombs" and the new version was kept at version "30" to contain the impact of the change, and any actual changes around library versioning where postponed. And unfortunately we haven't correctly followed this up since and it is indeed a blemish.

---

So in that sense I'm to blame for this inconsistency - best intentions paving the path to hell and all. This could've been avoided if plugins had never directly linked with libobs to begin with (and used a more stable "plugin API" facet), but alas...

I still believe that it is bad practice to make it this easy for users to install a plugin that is happily loaded by the dynamic library loader only to crash the application in unforeseen and "random" ways, no matter for how long this has been the status quo.

[mihawk90]

What won the argument for the time being (again, IIRC) was that we should not publicly announce such a change in the same version that we introduce it, which is very much a valid point.

…

I still believe that it is bad practice to make it this easy for users to install a plugin that is happily loaded by the dynamic library loader only to crash the application in unforeseen and "random" ways, no matter for how long this has been the status quo.

As the guy who kicked off the discussion that ultimately ended in reverting the change, I'll just say it here again for posterity:

Generally I absolutely agree with your viewpoint on this, and this is also what I reiterated in the discussion. The change absolutely should happen. The only thing I had an issue with at the time was the communication of the change (or rather, lack thereof). It was essentially a sidenote in the release notes bricking the entire Linux plugin ecosystem. And it was - as you said - the first such announcement (unless one follows the PRs as they come up - which I do, and still missed it) with the introduction of the change.

But yes, I do think it should happen, but there should be a reasonable leadup time for plugin developers - and potentially for the plugin template, which was another point of contention.