========================
Pootle 2.8 release notes
========================

*Released on 16 August 2017*

Welcome to Pootle 2.8!

If you want to try it, check one of the following:

- :doc:`Installation instructions </server/installation>`
- :doc:`Upgrading instructions </server/upgrading>`


Changes in Requirements
=======================

- Django>=1.10.5,<1.11
- zlib1g-dev required for lxml
- MySQL support uses mysqlclient instead of MySQLdb
- django-transaction-hooks is no longer used


Major Changes
=============

- Undertook a security audit
- Improved editor performance
- Added pluggable search backend
- Pootle FS


Below we provide much more detail. These are by no means exhaustive, view the
`git log <https://github.com/translate/pootle/compare/2.7.6...2.8.0>`_ for
complete information.


Details of changes
==================

- MySQL now uses the ``mysqlclient`` database driver instead of ``MySQLdb``,
  this is Django's preferred database driver.
- Undertook a security audit
- JavaScript fixes addressing performance and memory leaks in the editor
- Improved editor performance
- Extensive review and fixes to RTL layout
- Added pluggable search backend
- Pootle FS:

  - Added admin UI to set up projects configuration and language mapping
  - CLI
  - LanguageMapper
  - FileMapper
  - Store de/serialization
  - Removed the ability to add new TPs from the admin UI for Pootle FS projects

- Refactored code for syncing stores
- Changed the default ``robots.txt``

  - It is now a static file
  - Now the site is ``allow`` by default

- Editor:

  - Suggestions:

    - Support rejecting/accepting suggestions with comments

      - Suggesters are emailed based on new
        :setting:`POOTLE_EMAIL_FEEDBACK_ENABLED` setting

    - Accept suggestion without "can review" permissions if submitted
      translation is 100% match.
    - Prevent sending suggestions matching a previously submitted suggestion or
      current translation.

  - Current unit is shown at the top of the unit list so it is easier to use
    the navigation bar and it doesn't jump so much.
  - Faster terminology matching yielding more results by using stemming
  - Special characters are depicted by symbols from a custom-built font:

    - Properly display whitespace as special character
    - Technical details of escape sequences are omitted in the output displayed
      to end users

  - Buttons to add special characters display helpful information when hovering
  - Errors fixed:

    - Fixed bugs in muting/unmuting checks
    - Fixed filtering translations by month
    - Fixed ``TypeError`` error when filter in editor gets no units
    - Fixed bug that prevented translators from clicking on context rows to
      edit those context units

  - Translation memory:

    - Display original and translations side-by-side for TM results
    - AmaGama Translation Memory is now queried using CORS
    - AmaGama is only queried if the source language is supported by amaGama

  - Perform all highlighting in the client
  - Clearly present plurals to translator
  - Allow selecting similar translations' text in editor
  - Force word wrapping on long strings with no spaces
  - Improvements on timeline
  - Cross-language translation is now restricted to admins as it heavily
    impacts performance and translators are unlikely to require it
  - Removed ability to clear language or project dropdown to prevent
    performance degradation.
  - Alternate source language translations are no longer displayed for
    anonymous users.
  - Editor is disabled for users without the required permissions. Reviewing
    suggestions is similarly disabled without required permissions.
  - Check categories can now be used in dropdown to filter units.
  - Auto-matched translations are now highlighted as such.
  - Incomplete plural translations may now be submitted.
  - String error reporting form is now clearer and prevents empty reports from
    being submitted.

- Added suggestion bulk management:

  - Provides filtering by user, etc.
  - Allows to review multiple suggestions at once and reject/accept them at
    once optionally providing a comment for the suggesters

- Configuration system
- Plugin framework
- Comment system
- Removed Plurr format checks
- Removed `ENChecker`
- Added ``pootle_format`` app

  - Added support to have several formats in the same project

- Browse pages:

  - Refactored stats backend:

    - Removed stats refresh notice
    - Faster stats retrieval

  - Hide disabled items and allow showing them to admin users
  - Altered order of columns to highlight latest activity and pending work

    - Last updated data is now only shown to admin users

  - Altered order in which some items are listed by default:

    - Projects and languages are sorted by most recent translators changes to
      highlight activity
    - Virtual folders are sorted by priority to highlight most important
      strings to translate

  - Hid most of the special templates language data as it is unnecessary
  - Got rid of fat cookies:

    - Increases responsiveness and removes security issue
    - Most data is now stored in user session instead
    - Sidebar is no longer automatically open for anonymous users when an
      announcement changes.

  - Show leaderboard on top panel and expanded stats panel
  - Numbers are rendered in a locale aware fashion
  - Search widget is disabled if user cannot translate
  - Improvements to the statistics table for overly long filenames and smaller
    screens
  - Files dropdown no longer keeps references to empty directories
  - Fixed issue where "Back" button not always worked
  - Fixed issue with project dropdown when there are projects without name

- Search:

  - No longer autocompletes
  - Old 'Exact Match' was separated into 'Case-sensitive match' and
    'Phrase match' allowing finer-grained searches.

- Added team page:

  - Only for languages so far, and only available to language managers
  - Replaces permissions with roles
  - Provides direct access to suggestion bulk management

- Revamped user profile page

- Removed for performance reasons:

  - Removed statistics from user profiles. Will be brought back in the future.
  - Removed export view
  - Removed unperformant "More stats" in admin dashboard

- Removed reports feature
- Own Pootle localization changes

  - Updated translations. You can still `contribute translation updates for
    your language <http://pootle.locamotion.org/projects/pootle/>`_.
  - Now `compilejsi18n
    <http://django-statici18n.readthedocs.io/en/latest/commands.html#compilejsi18n>`_
    is used to compile JavaScript translations into assets, thus requiring
    `django-statici18n
    <http://django-statici18n.readthedocs.io/en/latest/index.html>`_ app.
  - Password reset email is now localizable in Pootle
  - Multiple changes in localizable strings to ease translation
  - Now Pootle bundles Select2 localization.

- Upload and download:

  - Disabled upload for non-PO projects as conflict handling currently only
    works in PO
  - Admins can upload translations as other user
  - Fixed error for stores with no revision
  - Added ability to download TMX exports

- New Machine Translation providers:

  - `Caighdeán <https://github.com/kscanne/caighdean/blob/master/API.md>`_ -
    Irish
  - `Welsh <http://techiaith.cymru/api/translation/?lang=en>`_

- Refactoring of models to increase performance, including dropping unnecessary
  indices.

- User input is sanitized for outgoing emails
- Usernames using latin1 characters are now allowed
- Improved RQ usage and new management commands
- Changed Pootle logo and styling
- Added the ability to use a custom logo with :setting:`POOTLE_CUSTOM_LOGO`
- Documentation updates


Pootle FS
---------

:ref:`Pootle FS <pootle_fs>` enables synchronization of Pootle against a
filesystem, or version control system, handling conflict resolution and other
situations of two files being out of sync.


Development changes
-------------------

- Updated and pinned PyPI requirements:

  - From now on requirements will be pinned in order to simplify support and
    development

- Tests:

  - Massive improvement in test framework
  - Coverage increased from 55% to 94%
  - Moved to tox
  - Travis caching and optimisations
  - Added JavaScript testing

- Code sanity:

  - Python code cleanup/linting pep8/pyflakes/pep257 to increase code health
  - Javascript code linting and cleanups
  - CSS code linting and cleanups

- Code polishing:

  - Moved all commands to argparse
  - Moved shortcuts to Mousetrap
  - JS improvements, move to React components

- Triage meetings are now held on a weekly basis


Command changes and additions
-----------------------------

- Running Pootle commands using :command:`manage.py` is no longer supported,
  use :command:`pootle` instead.
- :command:`pootle` warns if configuration is missing.
- :djadmin:`run_cherrypy` has been removed.
- :command:`start` has been removed, use :djadmin:`runserver` instead.
- :djadmin:`verify_user` and :djadmin:`purge_user` now accept multiple
  usernames.
- :djadmin:`refresh_scores` now recalculates user scores and accepts
  multiple usernames. It can be run by projects and/or languages.
- :djadmin:`contributors` command has been refactored in order to return more
  accurate results and has new options
  :option:`--since <contributors --since>`,
  :option:`--until <contributors --until>` and
  :option:`--mailmerge <contributors --mailmerge>`. The ``--from-revision``
  option has been removed.
- :djadmin:`list_serializers` has been added to view serializers and
  deserializers installed on your system.
- :djadmin:`flush_cache` flushes ``default``, ``redis`` caches,
  accepts :option:`--rqdata <flush_cache --rqdata>`,
  :option:`--django-cache <flush_cache --django-cache>` options.
- :djadmin:`config` was added to get, set, list, append and clear configuration
  settings.
- :djadmin:`init_fs_project` was added.
- :djadmin:`set_filetype` was added.
- Removed :command:`refresh_stats` and :command:`clear_stats` commands.
- :djadmin:`export` is now able to export zipped TMX files per translation
  project with the :option:`--tmx <export --tmx>` option.
  :option:`--rotate <export --rotate>` option allows old files to be removed.
- Added :djadmin:`schema` command that allows to dump the database schema on
  MySQL which is useful for diagnosing differences in database schema.
- Added :djadmin:`update_data` command that allows to update the stats data.
- :djadmin:`init` now creates a development configuration with
  :option:`--dev <init --dev>` option.


Changes in settings
-------------------

- MySQL database connections should now use ``STRICT_TRANS_TABLES``.
- :setting:`POOTLE_SCORE_COEFFICENTS` has been removed and replaced with
  :setting:`POOTLE_SCORES`.
- :setting:`POOTLE_SCORES` accepts custom settings for user scores calculation.
- :setting:`POOTLE_TM_SERVER` no longer receives the ``MIN_SCORE`` parameter,
  as it was misleading and had questionable effects.
- :setting:`POOTLE_TM_SERVER` now accepts a ``MIN_SIMILARITY`` parameter, to
  filter out results which might be irrelevant. To learn more, check the
  documenation on :setting:`MIN_SIMILARITY <POOTLE_TM_SERVER-MIN_SIMILARITY>`.
- :setting:`POOTLE_SEARCH_BACKEND` was added, to allow configuring the search
  backend to be used.
- Changed the default value for `ACCOUNT_SESSION_REMEMBER
  <https://django-allauth.readthedocs.io/en/latest/configuration.html>`_ so now
  sessions are always remembered.
- :setting:`POOTLE_EMAIL_FEEDBACK_ENABLED` was added, to allow disabling
  sending emails to suggesters when suggestions are accepted or rejected.
- Added new :setting:`POOTLE_CUSTOM_LOGO`, :setting:`POOTLE_FAVICONS_PATH`,
  :setting:`POOTLE_FS_WORKING_PATH` and
  :setting:`POOTLE_CANONICAL_URL` settings.
- Deprecated ``POOTLE_QUALITY_CHECKER`` setting.
- Added new :setting:`POOTLE_SQL_MIGRATIONS` setting.
- :setting:`POOTLE_MARKUP_FILTER` defaults to ``'markdown'``, and ``None``,
  ``'html'``, ``'textile'`` and ``'restructuredtext'`` values have been
  deprecated. Deployments using any deprecated markup must migrate manually to
  Markdown. This setting will be removed in the future since Markdown will be
  the only available markup.
- Added new :setting:`AMAGAMA_SOURCE_LANGUAGES` setting.


Credits
=======

This release was made possible by the following people:

Ryan Northey, Dwayne Bailey, Julen Ruiz Aizpuru, Taras Semenenko, Leandro
Regueiro, Igor Afanasyev, Claude Paroz, Safa Alfulaij, Rene Ladan, Kevin
Scannell, Jason P. Pickering, Eamonn Lawlor, Alexander Lakhin, Robbie Cole,
Rhoslyn Prys, Prasasto Adi, Nootan Ghimire, Mikhail Paulyshka, Mike Robinson,
leonardcj, Henrik Feldt, Francesc Ortiz, Allan Nordhøy, Christian Lohmaier,
Burhan Khalid, benbankes, Arash Mousavi, Andy Kittner, Adam Chainz.

And to all our bug finders, testers and translators, a Very BIG Thank You.
