Merge branch 'update-docs' into 'belenos'

Update docs

Update docs in order to follow the current workflow

Some ideas taken from the joind.in project

Video showing the process is avaliable here:
https://asciinema.org/a/587sed1f4fi0cj5lia2p9uoi5

See merge request !109
This commit is contained in:
Ruben Rodriguez 2015-06-25 15:42:58 +00:00
commit 817731df04
5 changed files with 136 additions and 447 deletions

1
.gitignore vendored
View file

@ -1,3 +1,4 @@
repos
jails
logs
PACKAGES/

112
CONTRIBUTING.md Normal file
View file

@ -0,0 +1,112 @@
# Contributing
Whether you've got a bugfix, documentation update, or new feature for us, these are the steps to follow to contribute code back into the main trisquel repo.
## 1. Prerequisites
1. Install needed packages for both, sources and binary builds
```
sudo apt-get install dpkg-dev devscripts git pbuilder quilt patch sed rpl parsewiki
```
1. Configure build environment for binary packages
```
git clone https://devel.trisquel.info/trisquel/trisquel-builder.git
ln -s $(readlink -f trisquel-builder/pbuilderrc) ~/.pbuilderrc
sudo ln -s $(readlink -f trisquel-builder/hooks) /var/cache/pbuilder/hooks.d
```
1. Create the build environment for each distribution/architecture you want to work with
```
sudo BUILDDIST=belenos BUILDARCH=amd64 pbuilder create
```
## 2. Get the latest code from gitlab
You'll need to understand a little bit about how git and gitlab work before this step (GitHub works the same way, but we like free software). In simple terms, log in [here](https://devel.trisquel.info/users/sign_in?redirect_to_referer=yes), visit the [trisquel/package-helpers project page](https://devel.trisquel.info/trisquel/package-helpers), and click the "fork" button to create your own copy of the repo. You will push your changes to this new repo under your own git account, and we will pull changes into the main repo from there.
For the sake of the rest of the examples in this guide, we're going to assume your gitlab username is "**richardtorvalds**" and you will be working with the "**hello**" package, and use those in our examples.
Now, we want to grab the latest from this newly created repository and pull it down to your local machine. Getting the latest code from your repo is simple, just clone it:
```bash
git clone https://devel.trisquel.info/richardtorvalds/package-helpers.git
cd package-helpers
```
This will give you a directory called "package-helpers" on your local machine with the latest checkout from your fork of the main package-helpers repo. **Note:** this is *not* a direct reference to the main trisquel repo. When you make changes in your fork, you'll need to let us know about it so we can pull it over....but that's later in the process.
## 3. Add a remote for the main package-helpers repo
One thing you'll need to do to make things easier to integrate and keep up to date in your fork is to add the main repo as a remote reference. This way you can fetch the latest code from the production version and integrate it. So, here's how to set that up:
```bash
git remote add upstream https://devel.trisquel.info/trisquel/package-helpers.git
git remote (this will list out your remotes, showing the new one we added)
```
Then, when you need to pull the latest from the main trisquel repo, you just fetch and merge the master branch:
```bash
git fetch upstream
git merge upstream/belenos
```
You can also use *git pull upstream belenos* if you want it all in one step.
## 4. Making a branch for your changes
When adding features or bug fixes, please create a separate branch for each changeset you want us to pull in, either with the issue number in the branch name or with an indication of what the feature is (feature, bugfix...).
```bash
git branch (lists your current branches)
git checkout -b bugfix-hello (makes a new branch called bugfix-hello)
```
## 5. Building the source package
If you want to create a new package, apache2 is a good starting point, so:
```
cd helpers
cp make-apache2 make-hello
```
For importing free packages from ppa's or other sources, check [make-toxcore](https://devel.trisquel.info/trisquel/package-helpers/blob/belenos/helpers/make-toxcore) and update the _EXTERNAL_ , _SIGNKEY_ and _changelog_ lines with your own values.
Then, run the helper with
```
bash make-hello
```
If everything goes fine, you will have your new source package ready at _PACKAGES/hello/_
## 6. Build and test the binary package
The last step generated a source package file, so we need to build the binary one:
```
sudo BUILDDIST=belenos BUILDARCH=amd64 pbuilder build PACKAGES/hello/*.dsc
```
The binary packages will be avaliable at _/var/cache/pbuilder/jenkins-repos/$BUILDDIST/_
## 7. Push your code and make a pull request
When you have finished making your changes, you'll need to push up your changes to your fork so we can grab them. With them all committed, push them:
```bash
git push origin bugfix-hello
```
This pushes everything in that branch up. Then you can go back to your forked package-helpers gitlab page and issue a pull request from there. Tell us what you want us to merge and what it does/fixes, and one of us will pick it up.
That lets us know that there's something new from you that needs to be pulled in. We'll review it and get back to you about it if we have any questions. Otherwise, we'll integrate it and let you know when it's in!
Hope this guide helps you get started in contributing to the trisquel project! If you still have questions, don't hesitate to join us on IRC - we're in #trisquel-dev on freenode -, or send a mail to the development mailing list trisquel-devel at listas.trisquel.info.

View file

@ -1,6 +1,6 @@
## Introduction
This set of scripts are helpers that modify and compile those packages coming
This set of scripts are helpers that modify and create the source packages coming
from the Ubuntu upstream which need it. It might be because they contain
non-free stuff, references to Ubuntu that need to be changed, or because we
want the package to work our way.
@ -9,20 +9,29 @@ This helpers are similar to some of those in the [gNewSense](http://www.gnewsens
some ideas and even some lines from them. If you plan to build an Ubuntu
derivative of your own, we suggest you to use Builder instead of this helpers.
All packages in this list are in the appropriate blacklist in the repository
updater, so they never enter into the repo from upstream and need to be
compiled with this helpers and pushed into reprepro. This helpers also need
to be run by hand - and the results tested - any time the repo watchdog warns
about pending updates from upstream.
Once a new package is added, it takes priority over the original one from Ubuntu,
so they never enter into the repo from upstream and need to be
compiled with this helpers and pushed into reprepro.
To add a package to the list, just copy one - `make-apache2` is a good template -
and rename it to `make-sourcePackageName`. To send the file back to us, or to
include any modification into the current scripts, use `bzr diff` and send
the output as an attachment to trisquel-devel@listas.trisquel.info. You need
to join the mailing list to send messages to it.
To add a package to the list, follow the [CONTRIBUTING](https://devel.trisquel.info/trisquel/package-helpers/blob/belenos/CONTRIBUTING.md) guidelines.
## Steps
Those are the steps done by the helpers:
1. Create local apt configuration, so you don't need to be root to run the helpers
2. Get the ubuntu and trisquel gpg keys
3. Get the source packages from their original repo
4. Uncompress them
5. Apply the changes described in the helper
6. Re-package it, adding "triquel$VERSION" version string
## Recommendations
* You don't need to use sudo in order to run those scripts, but some extra packages are needed:
`sudo apt-get install dpkg-dev sed git rpl devscripts quilt patch`
* Take care to use the right sourcePackageName, many source packages produce
several binary packages. `apt-cache showsrc binary-package` can help you.
* If possible, use sed to replace chains in the upstream source without the
@ -32,9 +41,8 @@ it at the `DATA/sourcePackageName` directory
would actually be shown to the user. Avoid replacing copyright statements!
* Try to write your replacements in a way they might work in future versions
of the upstream package. Well written regexps and sed will help with that.
* Some of this packages require the lsb to match Trisquel values. Edit the
`/etc/lsb_release` accordingly, or run the helpers in a Trisquel box.
* You can - and maybe should - run this scripts inside a chroot.
* You can test your changes by doing them inside the _PACKAGES/sourcePackageName/source/_ directory,
and running `dpkg-source -b .`, before being added to the helper script
## Netinstall
@ -54,3 +62,4 @@ To generate the images, we run the following scripts:
Then we push the results into the Trisquel repository and run the script
`make-debian-installer` to build the final images.

View file

@ -1,278 +0,0 @@
#!/bin/bash
#
# Copyright (C) 2011 Ruben Rodriguez <ruben@trisquel.info>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
set -e
export LD_PRELOAD="${LD_PRELOAD:+$LD_PRELOAD:}/usr/lib/libeatmydata/libeatmydata.so"
export DEBIAN_FRONTEND=noninteractive
MEM=$(cat /proc/meminfo |grep MemTotal| awk '{print $2}')
GPGKEY=8D8AEBF1
for VAR in LANG LANGUAGE LC_ALL LC_TIME LC_MONETARY LC_ADDRESS LC_TELEPHONE LC_MESSAGES LC_NAME LC_MEASUREMENT LC_IDENTIFICATION LC_IDENTIFICATION LC_NUMERIC LC_PAPER LANG
do
unset $VAR
done
export LANG=C
export HOME=/root
export DATE=$(date +'%a, %d %b %Y %T %z')
cd $(dirname $0)
if [ $# -lt 1 ]
then
echo "Usage: $0 package"
echo example: $0 linux lucid
exit 1
fi
PACKAGE=$1
CODENAME=$(grep 'export CODENAME=' helpers/config |sed 's/.*=//')
UPSTREAM=$(grep 'export UPSTREAM=' helpers/config |sed 's/.*=//')
WORKDIR=tmp/makepackage
REPOSITORY=$PWD/repos/$CODENAME
HELPERS=$PWD/helpers
for REPO in $(git branch|sed 's/.* //;/master/d')
do
[ -d $REPOSITORY ] && continue
mkdir -p repos/$REPO/conf repos/$REPO/incoming
cat << EOF > repos/$REPO/conf/distributions
Origin: Trisquel
Label: Trisquel
Suite: $REPO
Version: 42
Codename: $REPO
Architectures: i386 amd64 source
Components: main
UDebComponents: main
DebIndices: Packages Release . .gz .bz2
UDebIndices: Packages . .gz .bz2
DscIndices: Sources Release .gz .bz2
Log: $REPO.log
Description: Trisquel GNU/Linux packages for the $UPSTREAM based release
EOF
if gpg -K | grep -q $GPGKEY
then
echo "SignWith: $GPGKEY" >> repos/$REPO/conf/distributions
fi
cd repos/$REPO
reprepro -v export
cd ../..
done
disablescripts(){
# Disable service starter scripts
for i in /usr/sbin/invoke-rc.d /sbin/start /sbin/start-stop-daemon
do
mv $CHROOT/$i $CHROOT/$i.real
cp $CHROOT/bin/true $CHROOT/$i
done
}
createjail () {
[ -d jails ] || mkdir jails
CHROOT=jails/$UPSTREAM-$JAILARCH
C="chroot $CHROOT"
MIRROR=http://mir1.ovh.net/mirrors/ftp.ubuntu.com/ubuntu
mkdir $CHROOT-tmp
mount -t tmpfs -o size=500M none $CHROOT-tmp
debootstrap --arch=$JAILARCH $UPSTREAM $CHROOT-tmp $MIRROR
fuser -k $CHROOT-tmp
cp -a $CHROOT-tmp $CHROOT
umount $CHROOT-tmp
rm $CHROOT-tmp -rf
echo "127.0.0.1 localhost" > $CHROOT/etc/hosts
mount none $CHROOT/proc -t proc
mount none $CHROOT/dev/pts -t devpts
disablescripts
if gpg -K | grep $GPGKEY -q
then
cp /root/.gnupg $CHROOT/root -a
else
echo GPG private key for $GPGKEY not found, the packages and repositories will not be signed.
fi
echo $UPSTREAM-$JAILARCH > $CHROOT/etc/debian_chroot
cat << EOF > $CHROOT/etc/apt/sources.list
deb $MIRROR $UPSTREAM main universe
deb $MIRROR $UPSTREAM-updates main universe
deb $MIRROR $UPSTREAM-security main universe
deb $MIRROR $UPSTREAM-backports main universe
deb-src $MIRROR $UPSTREAM main universe
deb-src $MIRROR $UPSTREAM-updates main universe
deb-src $MIRROR $UPSTREAM-security main universe
deb-src $MIRROR $UPSTREAM-backports main universe
EOF
cat << EOF > $CHROOT/etc/apt/apt.conf.d/90recommends
APT::Install-Recommends "0";
APT::Install-Suggests "0";
EOF
export DEBIAN_FRONTEND=noninteractive
[ $UPSTREAM != hardy -a $UPSTREAM != lucid -a $UPSTREAM != maverick ] && echo "force-unsafe-io" > $CHROOT/etc/dpkg/dpkg.cfg.d/02apt-speedup
$C apt-get update
[ $UPSTREAM != hardy ] && $C apt-get --force-yes -y install sysv-rc dpkg sysvinit-utils upstart
disablescripts
$C apt-get --force-yes -y dist-upgrade
$C apt-get --force-yes -y install devscripts build-essential liburi-perl python-setuptools pkgbinarymangler wget rpl aptitude quilt fakeroot ccache
$C apt-get clean
wget -o /dev/null -O $CHROOT/tmp/key.gpg http://archive.trisquel.info/trisquel/trisquel-archive-signkey.gpg
$C apt-key add /tmp/key.gpg
# Hack for i386
if [ $JAILARCH = "i386" ]
then
for BIN in /bin/uname /usr/bin/arch
do
[ -f $CHROOT/$BIN ] || continue
mv $CHROOT/$BIN $CHROOT/$BIN.orig
cat << EOF > $CHROOT/$BIN
#!/bin/bash
if [ \$# -eq 0 ]
then
$BIN.orig | sed s/x86_64/i686/g
else
$BIN.orig "\$*" | sed s/x86_64/i686/g
fi
EOF
chmod 755 $CHROOT/$BIN
done
fi
umount $CHROOT/proc $CHROOT/dev/pts
}
prepare(){
[ -d jails/$UPSTREAM-$JAILARCH ] || createjail
cat << EOF > jails/$UPSTREAM-$JAILARCH/tmp/update
mount -t proc none /proc
export DEBIAN_FRONTEND=noninteractive
export LANG=C
apt-get update
apt-get --force-yes -y install sysv-rc dpkg sysvinit-utils upstart
for i in /usr/sbin/invoke-rc.d /sbin/start /sbin/start-stop-daemon
do
mv \$i \$i.real
cp /bin/true \$i
done
export DEBIAN_FRONTEND=noninteractive
apt-get --force-yes -y dist-upgrade
umount /proc
EOF
chroot jails/$UPSTREAM-$JAILARCH bash /tmp/update
if [ -f jails/$UPSTREAM-$JAILARCH/CurrentlyBuilding ]
then
echo The $UPSTREAM-$JAILARCH jail appears to be running $(cat jails/$UPSTREAM-$JAILARCH/CurrentlyBuilding |grep Package: |sed 's/Package:\ //'), aborting.
exit 1
fi
CHROOT=jails/$UPSTREAM-$JAILARCH disablescripts
}
tmpumount(){
grep jails/$UPSTREAM-$JAILARCH/tmp /proc/mounts -q || return 0
umount jails/$UPSTREAM-$JAILARCH/tmp && return
echo ERROR: could not umount tmpfs at jails/$UPSTREAM-$JAILARCH/tmp
exit 1
}
tmpmount(){
[ $MEM -lt 16000000 ] && return
grep jails/$UPSTREAM-$JAILARCH/tmp /proc/mounts -q && tmpumount
echo $PACKAGE | grep -q "linux" && return
mount -t tmpfs -o size=20G none jails/$UPSTREAM-$JAILARCH/tmp
}
compile(){
[ -d jails/$UPSTREAM-$JAILARCH/$WORKDIR ] && rm -rf jails/$UPSTREAM-$JAILARCH/$WORKDIR/
tmpmount
cp -a helpers/ jails/$UPSTREAM-$JAILARCH/$WORKDIR/
cat << EOF > jails/$UPSTREAM-$JAILARCH/usr/local/sbin/makepackage-handler
#!/bin/bash
WORKDIR=$WORKDIR
PACKAGE=$PACKAGE
cd $WORKDIR
[ -d LOGS ] || mkdir LOGS
[ -d PACKAGES ] || mkdir PACKAGES
export PATH="/usr/lib/ccache:${PATH}"
bash -e make-$PACKAGE 2>&1 || exit 1
EOF
cat << EOF > jails/$UPSTREAM-$JAILARCH/etc/pkgbinarymangler/striptranslations.conf
enable: true
components: main
invalid_currentlybuilding: ignore
posuffix: translations
oem_blacklist: partner
EOF
cat << EOF > jails/$UPSTREAM-$JAILARCH/etc/pkgbinarymangler/sanitychecks.conf
enable: true
EOF
cat << EOF > jails/$UPSTREAM-$JAILARCH/etc/pkgbinarymangler/maintainermangler.conf
enable: true
invalid_currentlybuilding: ignore
EOF
cat << EOF > jails/$UPSTREAM-$JAILARCH/etc/pkgbinarymangler/maintainermangler.overrides
default: Trisquel GNU/Linux developers <trisquel-devel@listas.trisquel.info>
ignore_domains: trisquel.info sognus.com listas.trisquel.info gnu.org fsf.org
ignore_emails: ruben@trisquel.info
EOF
chmod 755 jails/$UPSTREAM-$JAILARCH/usr/local/sbin/makepackage-handler
if ! chroot jails/$UPSTREAM-$JAILARCH makepackage-handler /$WORKDIR/make-$PACKAGE
then
rm jails/$UPSTREAM-$JAILARCH/CurrentlyBuilding
exit 1
fi
find jails/$UPSTREAM-$JAILARCH/$WORKDIR/PACKAGES/$PACKAGE/ -maxdepth 1 -type f -exec cp {} $REPOSITORY/incoming/ \;
tmpumount
}
JAILARCH=i386 prepare
JAILARCH=amd64 prepare
JAILARCH=i386 compile
JAILARCH=amd64 compile
cd $REPOSITORY
for dsc in incoming/*.dsc
do
if gpg -K | grep $GPGKEY -q
then
debsign -k$GPGKEY $dsc
else
echo GPG private key for $GPGKEY not found, the packages and repositories will not be signed.
fi
reprepro -v -C main includedsc $CODENAME $dsc || reprepro -S admin -P optional -v -b . -C main includedsc $CODENAME $dsc
done
find incoming -name *.deb -exec reprepro -v -C main includedeb $CODENAME {} \;
find incoming -name *.udeb -exec reprepro -v -C main includeudeb $CODENAME {} \;
rm incoming/*

View file

@ -1,155 +0,0 @@
#!/bin/bash
#
# Copyright (C) 2009,2010,2011 Ruben Rodriguez <ruben@trisquel.info>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
#
#
# This script compares two Sources.gz files, and reports available updates
if [ $(git status -s | wc -l) != 0 ]
then
echo You have uncommited git changes, stopping
[ 1$1 = 1"test" ] || exit 1
fi
PAIRS="taranis-lucid toutatis-precise belenos-trusty"
CODENAMES="taranis toutatis belenos"
UPSTREAMS="lucid precise trusty"
[ 1$1 = 1"test" ] && TEST=echo
WD=$PWD
LOGS=$PWD/logs
[ -d $LOGS ] || mkdir $LOGS
export LANG=C
FILE=$(mktemp)
TMP=$(mktemp -d)
cd $TMP
echo "List of packages with available updates:
" > $FILE
listmirror (){
DIST=$1
COMPONENT=$2
echo -n Downloading Sources.gz from $MIRROR/dists/$DIST/$COMPONENT/source/Sources.gz >&2
wget $MIRROR/dists/$DIST/$COMPONENT/source/Sources.gz -q -O - | zcat | egrep dsc$ | cut -d" " -f 4|sed 's/\.dsc//'
echo " Done" >&2
}
update(){
MIRROR="http://archive.ubuntu.com/ubuntu"
for i in $UPSTREAMS
do
listmirror $i main >> $i
listmirror $i-updates main >> $i
listmirror $i-security main >> $i
listmirror $i universe >> $i
listmirror $i-updates universe >> $i
listmirror $i-security universe >> $i
done
for i in $CODENAMES
do
MIRROR="http://archive.trisquel.info/trisquel"
listmirror $i main >> $i
listmirror $i-updates main >> $i
listmirror $i-security main >> $i
MIRROR="http://devel.trisquel.info/trisquel/$i"
listmirror $i main >> $i
done
}
higher(){
LIST="$(echo $* |sed 's/:/./g; s/+/./g') EOL"
HI=$(echo $LIST | cut -d" " -f1)
LIST=$(echo $LIST | cut -d" " -f2-)
while [ "$LIST" != "EOL" ]
do
TEST=$(echo $LIST | cut -d" " -f1)
dpkg --compare-versions $HI lt $TEST 2>/dev/null && HI=$TEST
LIST=$(echo $LIST | cut -d" " -f2-)
done
echo $HI
}
check(){
for package in $PACKAGES
do
LIST=$(egrep ^${package}_ $DISTRO)
current=$( higher $LIST)
LIST=$(egrep ^${package}_ $UPSTREAM)
upstream=$( higher $LIST)
if dpkg --compare-versions $current lt $upstream 2>/dev/null
then
# echo Current version in $DISTRO: $current 1>&2
# echo Upstream version in $UPSTREAM: $upstream 1>&2
echo Current version in $DISTRO: $current
echo Upstream version in $UPSTREAM: $upstream
[ $current = "EOL" ] && continue
if [ $package = "grub2" ]
then
echo WARNING! grub2 packages need to be manually compiled!
echo
continue
fi
cd $WD
$TEST git checkout $DISTRO
$TEST git merge master
cd $TMP
echo Upgrading $current at $DISTRO with ${UPSTREAM}_$upstream 1>&2
echo Running makepackage $package $DISTRO at $(date) $LOGS/$DISTRO-$package-$(date +%Y%m%d).log
echo Running makepackage $package $DISTRO at $(date) $LOGS/$DISTRO-$package-$(date +%Y%m%d).log 1>&2
$TEST sh $WD/makepackage $package $DISTRO > $LOGS/$DISTRO-$package-$(date +%Y%m%d).log && echo -n Done! || echo -n FAILED!
echo " ($(date))"
echo
fi
done
}
update
for pair in $PAIRS
do
DISTRO=${pair/-*/}
UPSTREAM=${pair/*-/}
PACKAGES=$(git --git-dir=/home/systems/devel/.git ls-tree -r --name-only $DISTRO|grep helpers/make-|sed 's/.*make-//')
check >> $FILE
done
report () {
#address=trisquel-devel@listas.trisquel.info
address=ruben@trisquel.info
echo "Subject: Compiling updates
To: $address
$(cat $FILE)" | $TEST /usr/sbin/sendmail -f watchdog@devel.trisquel.info -h correo.sognus.com -t $address
}
grep -q makepackage $FILE && $TEST report
rm $FILE $TMP -rf