doniphon: ok. let's take http://dpaste.com/219155/ as a working base.
jesse: http://wiki.foresightlinux.org/display/DEV/a+typical+bootstrap+group ?
doniphon: yes.
doniphon: we are all used to use group recipes in a passive way ... i.e. we put there what we want and conary gets all glued.
doniphon: here we will do that too but we use it in a new fashion.
doniphon: we use the group recipe NOT to build a group (of built packages) but just to instrument rmake to built all the packages referenced there and by a given order
doniphon: the group as we are used to see it is just built at the end of the process.
eMBee: tis that different from telling rmake to rebuild all packages in a group?
doniphon: slightly.
doniphon: so, in this case, what we are saying is ... we have here a list of recipes that we want to get built with this flavors, and in the case where a package comes via a pkgSpec (i.e. from a recipe with a != name) we just ponbt the source origin.
eMBee: wheat does "comes via a pkgSpec" mean?
doniphon: rmake is smart enough to just from that (with nothing previously cooked in repo even) to sort build order correctly and do all the messy work for us.
doniphon: for p in ('gcc', 'gcc-c+', 'libgcc', 'libstdc+'): r.add(p, flavor='bootstrap,!cross', source='gcc')
doniphon: 'gcc-c+', 'libgcc', 'libstdc+' do not exist as separate :sources ...
eMBee: ah, hmm
doniphon: but they appear as buildReqs of stuff in that group
doniphon: so we need a way to tell rMake in advance that so that it canorder things from scratch right
eMBee: ok, you are just explainingthat in some cases the package has no ssource so the source needs to mbe mentioned explicitly
doniphon: yes. and this is mostly the use case.
doniphon: now for something totally different ...
eMBee: ah, but because the packages are listed as dependencies, rmake needs to know aboutthem, otherwise it would complain because the packages do not exist yet
doniphon: yup
* jesse raises hand. isn't the first step to create the chroot (within which we do all the cross-compiling)?
eMBee: so what is the difference to telling rmake to rebuild all packages in a group? is it that because the binary packages do not exist yet, rmake by itself figures outthat it needs to build the packages before it can build the group?
doniphon: yup
doniphon: jesse: on step 1 - chroot is made of 'donnors' stuff (be it rPL or FL2). from that you cross to a sysroot what you want, and so on. from step 2 up your chroot is fully native already
doniphon: now... recipe classes
eMBee: wait
doniphon: ok
eMBee: jesses question
doniphon: yes ?
eMBee: oh
jesse: so, how is the chroot created?
doniphon: rmake
doniphon: wait a sec
eMBee: donnors is donors?
doniphon: yep
doniphon: http://dpaste.com/219162/ <<
eMBee: so we are building the chroot from stuff that others have made
doniphon: in step 1 yes.
eMBee: the question is, where is the script/group that makes up the chroot?
doniphon: by parts...
eMBee: or the list of buildrequires?
doniphon: chroot is created as it is always. you point a group/label in context for resolution purposes and conary just fetches from there what we want it to want.
doniphon: so for gcc
doniphon: what we say is ...
doniphon: whatever gcc we are building we want 1st ... 'file:runtime', 'flex:runtime', 'gettext:runtime', 'perl:runtime', 'texinfo:runtime', 'byacc:runtime', 'bison:runtime',
doniphon: so - in step one those will come from the donnor
doniphon: donor
eMBee: ahh, so given the bootstrap group above, rmake starts building packages, and it simply fills the build requirements with packages from labels that we tell it to look at in our configuration
doniphon: right
jesse: so it's not a hand-made chroot, but one created by rmake
doniphon: note that 'file:runtime', 'flex:runtime', 'gettext:runtime', 'perl:runtime', 'texinfo:runtime', 'byacc:runtime', 'bison:runtime', will end being built later in this group so that this foreign reliance stops in step2
doniphon: yup
doniphon: there are no hand made stuff here
eMBee: so like we are building atoolshop, and we borrow our neighbors tools until we have our own 
doniphon: yup
doniphon: jesse: clear now ?
jesse: but do we later do 'rmake chroot' and run 'cvc cook' inside it?
doniphon: jesse: only for debugging purposes.
jesse: "What we do when we use rMake is 'just' to have rMake to populate a chroot's root from scratch with what we deem as buildReqs and run cvc cook inside it."
eMBee: that's a quote from last time btw
doniphon: oh
doniphon: that is what rmake always do. but that is an unattended process
doniphon: no need to be done manually
jesse: ok
jesse: got it
doniphon: more questions ?
jesse: no from me
doniphon: ok.
doniphon: now something different
doniphon: info-* packages and recipe classes
ermo: Oh, sorry, didn't realize we had started
ermo: 'we' he says
doniphon: ?
eMBee: ermo had this channel window hidden somewhere, not paying attention to it
* ermo reads scrollback – eMBee is correct
doniphon: we 've been getting better and better abstracting a lot of common actions via superclasses . and i'd like us to take that a lot further.
eMBee: can you cite some examples?
eMBee: kde,gnome?
doniphon: i'd like us to extend all the tricks we already do in say gnomepackage to all conary base classes
doniphon: (in a backward compatible fashion - so that old recipes work)
eMBee: like add new stuff to base class, old recipes ignore new stuff, new recipes use it...
doniphon: the idea is that if i have a AutoPackageRecipe i should not even have to have a unpack call... just a line saying archive = 'mirror://' and another saying patches = [] ... (as we do today with gnome / kde superclasses)
TForsman: i asume all superclasses can be used same way as gnome superclass?
doniphon: no, trickier
jesse: sounds good
doniphon: recent conary versions give us ways to override the default conary baseClasses
doniphon: so we can have there whatever we want
doniphon: we just need to keep classes naming consistent
TForsman: i mean, we can use same way to add patches in AutoPackageRecipe as we can do in Gnome recipe?
jesse: (actually with current gnomepackage, i hate it that in order to add an action to the install process, you have to call GnomePackage.install(r) manually)
doniphon: ultimatelly that is the goal
eMBee: wouldthis get us a common structure to handle autopackage vs cpackage for example? having to reorder things just to upgrade from cto auto is sometimes annoying
doniphon: yup
doniphon: that is the whole goal. more cosnistency and simplicity
eMBee: nice
TForsman: would be nice to get a more default in recipes superclasses
doniphon: BaseRequiresRecipe
doniphon: PackageRecipe
doniphon: BuildPackageRecipe
doniphon: CPackageRecipe
doniphon: AutoPackageRecipe
doniphon: ^^ those would be the ones that we would tweak
doniphon: syntax / mode of behaviour of those exactly same as superclasses. only difference is that we do not have to use loadSuperClass in the recipe to consume them as they are native to conary
eMBee: isthat hardcoded? or what makesthem native?
doniphon: so ... what i would like you guys to start thinkering is what high level commodities you consume in our superclasses that you would like to be able to use anywhere in any kind of recipe
TForsman: cpanpackage too? the weirdest recipes today, if you ask me. but easiest too
eMBee: can other classes be made native?
doniphon: yes.
doniphon: you just patch conary
eMBee: ah, ok, for the above tweaking, do we also needto patch conary?
doniphon: hg pull hg.rpath.com/conary-2.1/ ; more conary/build/packagerecipe.py
doniphon: eMBee: no.
eMBee: ah, interesting
doniphon: we can instrument conary to use ours and they will override stock ones. (made via a context call)
eMBee: aha
doniphon: http://dpaste.com/219166/ << something like this ...
eMBee: but for GnomePackageRecipe etc we still need loadsuperclass?
doniphon: you tell conary to forget what he tought he knew ... and you point location of what you want it to consume
eMBee: hmm, can't we add any kind of recipe to that list?
ermo: who will do QA on our 'custom conary plugins'?
doniphon: eMBee: depends if we want to patch conary or not. if we do is to avoid the loadSuperClass('gnomepackage') in all recipes that consume it
doniphon: we. QA will be easy to do really, as if this is busted - things simply stop building.
eMBee: we can continue this topic later, it's not so relevant
doniphon: ok.
doniphon: next info-files
* eMBee would rather go back to he original topic, sorry for sidetracking
doniphon: which a boring PITA that needs to be tracked
ermo: 2nd question: If we create a GnomeRecipe (assume that this class is valid) why doesn't it already load whatever superclass we need?
doniphon: if we add it to the default 'namespace' conary has with a list of classses to look at .. .we do not need to point it anymore
eMBee: but that requires patching conary?
doniphon: one liner.
eMBee: ok
doniphon: back to info-*. (will become clear why they are important)
ermo: so we tell conary that FooBarRecipe(GnomePackageRecipe) implies that it should loadSuperClass('gnomepackage') into the current recipe namespace by patching conary?
doniphon: yup
ermo: info-files: What happened to Mark__T's idea of """<METADATA>""" inside recipes?
doniphon: almost if not all other pkging systems handle users and groups data as 10th class citizens
doniphon: ermo: blocked upstream by how that is gonna be handled. groups code already working reasonably well for me so that will be looked again real soon (able from this week to get current 2-devel groups to build and minimally behave with new code)
doniphon: almost if not all other pkging systems handle users and groups as 10th class citizen
ermo: i.e. metadata = """<structured metadata in valid python syntax inside a python string>"""
doniphon: i know
ermo: doniphon: but the others here might not know?
doniphon: ok
doniphon: if a package needs a to create a user or a group... traditional way is to... create it again - no matter it exists or not ...
eMBee: unless it has some smarts to detect that the user/group already exists?
doniphon: to make things messier ... even when ppl had agreed in the user/group names (for deamons / whatever) too many timrs those do not have a default uid / gid ...
* eMBee nods
* eMBee has been bitten by that
doniphon: so it is perfectelly possible to get a package that creates a user / group foo in 2 != RHEL boxes and end with wholly != uids / gids
doniphon: conary goes away with that by having users and groups as 1st class citizens that deserve their own packages
doniphon: why this matters now this early ... ?
doniphon: a few of the packages we are creating early deal with creating the system's baselayout ... and that includes a few stock users/groups ... root/wheel/etc
doniphon: since we do not want to cheat ... (and hack the group craetion in a postIstall script).
eMBee: is fl:2 cheating in this regard?
doniphon: what we need to do (and this is mostly a one time nuisancy) is to create our own uid/gid registry and stick to it
doniphon: no - but things a bit undisciplinated.
eMBee: please explain/elaborate
eMBee: how is fl:2 undisciplinated?
doniphon: some uids/gids changed upstream and we didn't followed. and some were randomly attributed because upstream diverges wildly
doniphon: also - we 're limited by what rPL has as we still consumea fair bit of them
eMBee: what does rPL have?
jesse: #FL-2300
foresight: <http://issues.foresightlinux.org/browse/FL-2300> properly document our {u,g}ID registry | Status: Open | Resolution: Unresolved | Updated: 8 months ago
eMBee: you mean their info packages?
doniphon: yup
doniphon: taht is what i'm talking about
doniphon: so - what i 'd like is - as we keep moving - is to have an updated table of users/ groups we consume... and the uids/gids we attribute to them. (by default those are the upstream ones ... RHEL5 1st, if not defined there Fedora, if not used / random in any of those - we define one)\
eMBee: ok, wonder if that is a topicfor linux standard base
ermo: doniphon: so that basically means that we need to have a running RHEL5 system and a running Fedora system 'for reference' at all times? Because if they change, they are still the reference?
doniphon: they do not change often
ermo: eMBee: Yeah, you have a point there...
doniphon: been there ; done that. there's an entity that handles that ... move way slowly ...
eMBee: ermo: naaah, we just need t look at their docs
ermo: eMBee: uh, if I'm not mistaken, /etc/group and /etc/password are the canonical sources here?
doniphon: in short - where they have uids/gid attribution defined for known values we want to follow them, where they don't we want to have our own sane values (and in fashion that obvioulsy do not collides with theirs)
doniphon: yup
doniphon: conary rq --install-label=foreplay.rpath.org@devel:id | grep :source << this is the set we consume for bootstrap purposes atm
ermo: ok, in that vein I can see a challenge when attempting to integrate Ubuntu AND Fedora stuff on the same system
doniphon: why ?
ermo: as their bases are definitely != in ever so many ways.
doniphon: yup
eMBee: tha's a case for linux standard base
ermo: for starters, they have a different concept/division of user UIDs and system UIDs.
ermo: In the case where we chose Ubuntu as Canonical Upstream (hehe), and the UID/GID collides with fedora, what do we do?
doniphon: ermo: we won't be able to fix them.. i just want us to be consistent
doniphon: ermo: doesn't matter
jesse: it can't be worse that how things are now
jesse: s/that/than
doniphon: as we use ours... mess is when they decide to change user/group names ,... (netdev comes to mind)
doniphon: anyway ... taht is a issue we already face so ..
ermo: doniphon: what was the issue with netdev?
jesse: doniphon: is the 'registry' simply a matter of a wiki page?
doniphon: yes. mostly. (and a set of sane info-* recipes)
doniphon: basically i want to have ppl knowing (or have a way to know) where everything pops from, and why
doniphon: so that things stop happening as of by miracle or voodu
eMBee: ok, can we move forward?
doniphon: yup
ermo: I have an objection to the way rPath use their registry. We can have my_uid/my_gid, my_uid/another's_gid, my_uid/no_gid. I would prefer it if my_uid was always equial to my_gid (if possible). This would of course leave holes (unused uids/gids) in the registry.
ermo: Is this a valid concern or should I just 'get over it'?
doniphon: ermo: the places where it is != there is beacause upstream has that too
ermo: kk
doniphon: the rest is cosmetics
doniphon: guys - can be here more 10m (other wise won't get lunch at the mall)
doniphon: questions plz ?
ermo: when will we see you back from lunch?
doniphon: 1 sec
ermo: uh, that's ... fast?
doniphon: btw ... i'll be in vacation 23/30 July. i expect the week after to start the hands-on thing. i.e. i'll have cleaned from cruft that would only confuse what i have here and we 'all start consuming/reviewing it.
Page Operations
Browse Space