Project Handbook

From Askmonty.org

Main Page >  About Us 
Company rules | Employment | The hacking business model | Privacy Policy | Project Handbook | Working with the community

This is the Monty Program Ab Project Handbook. It is especially relevant to how we work on MariaDB. It is on the public wiki to benefit all MariaDB developers, but is primarily intended to serve Monty Program and its projects. However, other MariaDB contributors may wish to align their practices and schedules to this handbook.


Contents

Introduction

Q & A about Project Management

Q: Do we really need project managers? Isn't it best the developers talk to customers directly?

It is good for the developer to talk to the end user directly. However, it would be a waste of the developers time to do all the things a Project Manager does.

Things the Project Manager does:

  1. Participate in the negotiation phase before the actual project.
  2. Govern that the development effort is consistent with the contract agreed with the customer (or just the internal project plan, when there is no external customer).
    1. For instance, the Project Manager may decide to redefine the scope of a project (ie drop a feature) to stay within agreed costs or meet a deadline.
    2. Note that also the customer often has a designated contact person with authority to give directions, even if engineers from both sides may otherwise be in frequent contact with each other.
  3. Handle most of the customer communication.
  4. Protect the developer from the customer.
    1. Allow the developers to focus on what they're doing.
    2. A customer may try to throw more work at the developers, a Project Manager will handle this so it is done properly with updated contracts and project plans.
  5. Help the developer with any obstacles that may turn up.

Things the Project Manager does not do:

  1. The project manager is not the technical leader. This means the project manager can not force technical decisions on the developers, he is at most involved in discussions or may indirectly influence them by reminding about a cost or time limit for the project.


All in all, the Project Manager is kind of like a secretary to the developer. Keeps track of things, handles the correspondance and in general helps the developer be more productive.

Q: Whatever happened to "release when it is ready"?

Various answers include:

  1. I have no idea, but I think Debian is the only project left doing it. Everyone is all about time-based releases now.
  2. Even if we did "release when it is ready", how do we determine when that is?
    1. This Project Handbook should answer questions like release criteria, and who makes the decision to push, release etc...
  3. Unfortunately, as we do projects for paying customers we don't have that luxury.
  4. Scrum brings an interesting new philosophy to this by acknowledging that the world is not black and white, and requirements may even change while you're working. For instance a customer may be fine with dropping some of the goals and in return get the benefit of the new feature faster. Hence the strive to keep the product always in a "potentially releasable" state.

About Scrum

We used to have: Waterfall model: Extensive planning up-front, then work to meet your deadlines.

Now we have: Scrum: Agile and iterative, less planning.

The Open Source development process is a creature of its own and not directly related to either of the above. Open Source has always been quite agile. Even so, we try to adopt Scrum when beneficial.

Scrum is based on the following principles:

  1. To think that a complex project can be defined and estimated in advance is an illusion.
  2. A project starts with a product vision and a prioritized list of features (backlog).
  3. Serious planning is only done at the time when a task is selected to be worked on.
  4. The project is divided into sprints. Sprints are timebounded (usually 30 days). Each sprint ends in release of a new milestone.
  5. A sprint starts by assembling a sprint backlog, a subset of list items to be developed during the sprint. Only then are the items defined more closely (HLS, LLD)
  6. A team is given relative autonomy during the sprint, the Scrum Master is part of the team.
  7. During the sprint, tasks may be redefined or dropped with an eye to finish on the given date.
  8. The sprint ends by demonstrating a working milestone release.
  9. Even first milestone releases should do something useful (agile approach).

Where is the management part of such projects? Note the emphasis on time-bounded milestones.

The daily cycle of Scrum is famous for the Scrum meeting with the 3 questions:

  1. 15 minute team meeting each day
    1. What have you done since the last Daily Scrum regarding this project?
    2. What will you do between now and the next Daily Scrum meeting regarding this project?
    3. What impedes you from performing your work as effectively as possible?
  2. pigs and chickens = spectators welcome but silent
    1. This is also the main reporting channel

Some things not compatible with an Open Source and/or Virtual Environment way:

  • Emphasis on open office space.
  • Physical meetings and whiteboards.
  • Despise written communication.


Good news & things to improve

  1. We already do the Engineering Process part very well! Coding style, Reviews, automated builds, automated QA... (Only needs to be explained to me and documented.)
  2. Scrum was introduced into MySQL in 2003!
  3. ...but we probably can improve or refine our practices at the Project Management level.
  4. As a company we are still only building many things on the Business Management level.

The 3 levels of Project Management

  1. Business Management level
    1. Negotiation, contracts, product vision...
  2. Project Management level
    1. Planning, risk management, weekly meetings, customer communication...
  3. Engineering Process level
    1. Design, Code review, QA...


Project Handbook

Business Management

General

The business management level is concerned with the question: Why' do we do a project?

  • Who benefits from a new feature and how much?
  • Are there other solutions, workarounds, competitors?
  • What is the project cost and duration?

In a customer project, it is the customer who answers the above questions, whereas we ask ourselves:

  • Is the customer paying enough to cover the costs and some profit?
  • When will we have the right resources available?
  • What is the complexity and risk level of the project? (And how do we counter that?)
  • Is the contract such that we can commit to it (may have all kinds of requirements)?

Processes

Negotiation with the customer is handled by a Sales Manager.

Owner of Sales: Ethan.

For any project engagement, following tasks are included:

Project scoping defines "what needs to be done" in a way that developers can work on the agreed tasks and estimates the project duration and complexity.

Owner: Project Manager (Henrik), working closely with an Architect or an Engineer.

Resourcing check, to verify that needed resources are available for the project.

Owner: Project Manager (Henrik). (Could be Engineering Manager.)

For any contract

  • The contract is an unmodified template previously approved by the COO.
  • If it is a custom contract or modifications were made, the COO approves it.
  • The Sales Manager, his manager, COO and CEO will have various approval levels to discount from list prices.

Type of project contract: Fixed deliverable & price

This contract type is used for relatively simple projects. The project can be defined and estimated by one or a few worklogs and executed by one or a few engineers.

A contract template for this type exists, and a contract is always accompanied by a worklog description.

Type of project contract: Scrum project

This contract type is used for longer and more complex projects. The duration of the project is more than 3 months and is done by a team of 2 or more engineers.

The contract does not fully define all details of the deliverable. Instead it defines:

  1. The overall goal/vision for the project.
    1. There may be a project plan, but it is not contractually binding.
  2. A list of tasks/features to be implemented in the project (the Scrum backlog). The list of items includes a first estimate of their duration/cost, even while they are not yet defined in great detail.
  3. The size/composition/allocation of the project team and project length in an amount of calendar months (sprints).
  4. The contract also defines how and at what price the customer can extend the project time with additional months/sprints or also how he can choose to terminate it earlier (typically paying a penalty fee, but less than the remaining amount for the full project).
  5. How the customer can monitor project process via our Scrum reporting mechanisms.

The contract is time bound. There is no commitment from Monty Program Ab to deliver to the specification, only to deliver a certain amount of man-hours. Note that if the project requires more detailed arhcitecture studies before coding, that can be included as tasks in the first sprints.

Otoh, the customer gets the following benefits:

  • Transparency: The team uses our Scrum process, and the customer can monitor progress in daily/weekly meetings, on IRC, mailing lists and worklogs.
  • Control: The customer has control over the project especially at the start and end of each sprint (as defined by Scrum).
  • Flexibility: Since the contract does not define a "deliverable set in stone", the customer can modify requirements as the project progresses.
  • Focus: The Project Manager is still responsible to guide the project in a direction that keeps focus to the vision and delivers value to the customer.
  • Flexibility: The customer can easily extend or even shorten the project.
  • Lower price: Since the risk to Monty Program is lower, the project price is lower than it were for a fixed price project.
  • Lower risk: Also the customer benefits from lower risk, because he can terminate the project if business reasons have changed.

Deliverables into next level

Either of the contract types above. External or internal customer. Project Manager and Team.

Project Management level

This level is concerned with working towards the project goal and communicating between the different project parties. Activities on this level include those to start and end the project and routines like a weekly meeting or reporting.

Most activities defined by Scrum happen on this level.

Teams

Scrum is based on teams that can work on autonomous units. A Team should include

  • various roles to be "complete", including Architect, QA, documentation, etc...
  • some roles may have to be part time, but Team Members need to be able to commit to the team.
  • a Scrum Master that leads the team in the Scrum activities. This may be the Project Manager, or a member of the team. A requirement is that the Scrum Master is involved with the teams work every day.
  • approximately 7 people.

Project startup tasks

Input from Business Management level is a contract (or internal project plan). Review the Business Management level section of this handbook and use it as a checklist that all bulletpoints are addressed (is risk management addressed anywhere?).

If contract is of the "Scrum project type" then this section defines the Monty Program Ab Scrum process.

If contract is a "fixed deliverable" type, then the project can hopefully still be included in the Scrum activities of a team (that may also do something else) and completed in 1-3 months. Alternatively it can be executed in traditional waterfall model outside the Scrum cycle. This handbook does not currently define that process. (But such projects should be short and manageable anyway.)

The project is then broken into a number of sprints. The tasks are not assigned to specific sprints beyond the first ones.

Sprint start

Scrum defines a Product Owner. This is usually the Project Manager. (Usually not the external customer.)

During the first 24 hours of a sprint, the Product Owner and the Team select tasks from the Product Backlog into a Sprint Backlog. This is the list of tasks that the team commits to finish during the Sprint. The Team spends most of this time refining the HLS and LLD of the tasks.

After the first 24 hours work is started on the selected tasks. If tasks are not perfectly defined yet, the HLS and LLD can be further refined during the Sprint.

Results of these activities are recorded into Worklog.

Daily/Weekly cycle

Scrum advocates a 15 minute daily (face-to-face) meeting, and working in the same location just in general. We adapt it to a virtual company here:

Weekly Scrum call

  • Each Team gathers into a Weekly call.
  • This should be 30 minutes, but if necessary is allowed to be longer.
  • Only issues related to this Team and the ongoing Sprint are discussed.
  • The Scrum format is used: each Team member answers the 3 questions in turn.
  • A summary of discussed issues is sent to the public maria-discuss list.
    • Note: Try Gobby for live notes. Failing that, try Google Docs?

Daily IRC checkpoint

  • In addition to the weekly call, the team decides on a daily checkpoint when all team members can be found on IRC.
  • The Scrum Master will decide if he wants to go through the formal Scrum questions, or if just "popping in" is enough. He may only have questions to some team members.
  • Each team will document the daily checkpoint time on its team page
  • If the Scrum Master asked members the formal Scrum questions, or if substantial issues were discussed in general, the Scrum Master will send the IRC log to the public maria-discuss list

Timezone management

  • The weekly call and daily IRC checkpoint provide times where the entire team is guaranteed to be online at the same time.
  • In addition the team should strive to arrange it's work so that the whole team has at least 4 hours overlapping online time.
  • If this is impossible, the team should strive to a rythm where each member has 4 hours overlapping with most others in the team, even if it's not the same 4 hours for everyone.

External participants

  • Anyone is able to follow the public proceedings of the Scrum meetings on #maria and maria-discuss.
  • In the spirit of "chickens" the Team is free to ignore "noise" on the IRC channel during the Scrum meeting if it feels it needs to focus on its own issues.
  • An active MariaDB contributor may however be incorporated into the Team and may participate in the Scrum meetings as a Team Member.
  • SIP access may be given to such contributors, as well as "chickens" such as a customer representative.

Company internal weekly reporting

A company internal weekly report should still be sent to the internal mailing list. However, any issues reported in the Scrum meetings can be omitted, so the email might be quite short. But you must always report your total hours worked per week to the internal list.

Sprint end

  • The team produces a milestone release at the end of the sprint.
  • The milestone release is a QA tested, "potentially releasable" increment to the product. This includes documentation, etc.
  • The focus is on producing new functionality during the sprint. It is better to produce 1 new feature than 2 features that are 50% ready.
  • The new functionality should be presented and described through some form of Internet communication. This is more than a downloadable release and might be done by producing blogs, benchmark results, Howto's, Knowledgebase articles, Youtube videos, whatever is appropriate.
  • During the last 24 hours of the sprint, management and customers will review the progress from the presentations and discuss it with the Team.
  • For the rest of the day the Team may finish any unfinished cleanup tasks, or spend on emails, surfing, educating themselves, or 20% projects.

Project end / release

  • Every sprint ends in a milestone release, but not every sprint ends in a real release.
  • Typically a project will end in a release of the product.
  • The contract must define clear acceptance criteria to the project deliverables. The Project Manager is responsible for securing such acceptance.
  • A successful project finished in time should be celebrated.
  • A project end (and often also milestone end) triggers actions back to the Business management level, such as invoicing and paying of bonuses.

TODO: Where are MariaDB release criteria defined? (Alpha, Beta, RC, GA) Can we link to them from here?

Engineering Process

Routines to document...

TODO: Most of these things we seem to do quite well. But they need to be documented here:

  • Howto write High Level Specification, High Level Design, Low Level Design and review them
  • Links to MariaDB/MySQL internals guide
  • Coding style guidelines? (if any)
  • Links to bzr and launchpad guides
  • BuildBot
  • Automated QA
    • Standard test harness
    • Test case to prevent regression on bug fixes
    • Performance regressions
  • Non-automated QA
  • Code review process
  • Committing, pushing into a branch, pushing into mainline
  • Documentation update
  • Release notes

Estimating and accounting hours

Many customer projects are billed by the hour. Even if not, it is of interest to management to follow how much time different projects or tasks have consumed.

Also estimates should be produced for all tasks, even when it is not part of a customer project. This is still helpful to prioritize tasks. It is also good practice so that you become better at estimating duration of your own work. (But it is ok to spend less time estimating on non-customer projects.)

We should strive to use Worklog to account for hours per task. Unless otherwise stated, hours should be reported and up to date in Worklog on Mondays when sending out the weekly report. We will work on enhancing this aspect of Worklog in the future.

For internal purposes the weekly report email must still contain the total hours worked (for paying your salary).

Retrieved from "http://askmonty.org/wiki/Project_Handbook"