From 347ab4d54a841041dab095d3439c5098dff5090f Mon Sep 17 00:00:00 2001 From: Roman Grundkiewicz Date: Mon, 5 Sep 2022 16:51:48 +0100 Subject: [PATCH] Upgrade dependencies in the documentation framework (#965) * Upgrade lxml to 4.9.1; set docutils no higher than 0.17 * Remove mistune<2.0.3 due to potential vulnerability * Fix badges in docs * Update CHANGELOG --- CHANGELOG.md | 5 +++-- doc/.gitignore | 1 + doc/Makefile | 3 ++- doc/conf.py | 13 ------------- doc/index.rst | 22 +++++++++++----------- doc/requirements.txt | 4 ++-- 6 files changed, 19 insertions(+), 29 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a6614909..439a0c3d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,17 +13,18 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. ### Fixed - Multi-loss casts type to first loss-type before accumulation (aborted before due to missing cast) - Throw `ShapeSizeException` if total expanded shape size exceeds numeric capacity of the maximum int value (2^31-1) -- During mini-batch-fitting, catch `ShapeSizeException` and use another sizing hint. Aborts outside mini-batch-fitting. +- During mini-batch-fitting, catch `ShapeSizeException` and use another sizing hint. Aborts outside mini-batch-fitting. - Fix incorrect/missing gradient accumulation with delay > 1 or large effective batch size of biases of affine operations. - Fixed case augmentation with multi-threaded reading. - Scripts using PyYAML now use `safe_load`; see https://msg.pyyaml.org/load - Fixed check for `fortran_ordering` in cnpy - Fixed fp16 training/inference with factors-combine concat method - Fixed clang 13.0.1 compatibility +- Fixed potential vulnerabilities from lxml<4.9.1 or mistune<2.0.3 ### Changed - Make guided-alignment faster via sparse memory layout, add alignment points for EOS, remove losses other than ce -- Negative `--workspace -N` value allocates workspace as total available GPU memory minus N megabytes. +- Negative `--workspace -N` value allocates workspace as total available GPU memory minus N megabytes. - Set default parameters for cost-scaling to 8.f 10000 1.f 8.f, i.e. when scaling scale by 8 and do not try to automatically scale up or down. This seems most stable. - Make guided-alignment faster via sparse memory layout, add alignment points for EOS, remove losses other than ce. - Changed minimal C++ standard to C++-17 diff --git a/doc/.gitignore b/doc/.gitignore index 4d192b77..47b72ab0 100644 --- a/doc/.gitignore +++ b/doc/.gitignore @@ -2,3 +2,4 @@ api build doxygen venv +CONTRIBUTING.md diff --git a/doc/Makefile b/doc/Makefile index 84310d9d..aa2048b8 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -14,10 +14,11 @@ help: # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile + cp $(SOURCEDIR)/../CONTRIBUTING.md $(SOURCEDIR)/ @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) # Clean target as recommended by Exhale # https://exhale.readthedocs.io/en/latest/usage.html#optional-create-a-proper-clean-target clean: - rm -rf doxygen/ api/ + rm -rf doxygen/ api/ $(SOURCEDIR)/CONTRIBUTING.md @$(SPHINXBUILD) -M clean "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/doc/conf.py b/doc/conf.py index b0c68bcd..192dd27d 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -114,16 +114,3 @@ exhale_args = { primary_domain = 'cpp' highlight_language = 'cpp' - -# A trick to include markdown files from outside the source directory using -# 'mdinclude'. Warning: all other markdown files not included via 'mdinclude' -# will be rendered using recommonmark as recommended by Sphinx -from m2r import MdInclude - -def setup(app): - # from m2r to make `mdinclude` work - app.add_config_value('no_underscore_emphasis', False, 'env') - app.add_config_value('m2r_parse_relative_links', False, 'env') - app.add_config_value('m2r_anonymous_references', False, 'env') - app.add_config_value('m2r_disable_inline_math', False, 'env') - app.add_directive('mdinclude', MdInclude) diff --git a/doc/index.rst b/doc/index.rst index d19bb4b0..9d769c32 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -1,7 +1,7 @@ Welcome to Marian's documentation! ================================== -|buildgpu| |buildcpu| |tests| |release| |license| +|ubuntu| |windows| |macos| |release| |license| Marian is an efficient and self-contained Neural Machine Translation framework with an integrated automatic differentiation engine based on dynamic computation graphs, written entirely in C++. @@ -19,7 +19,7 @@ This is developer documentation. User documentation is available at https://mari factors api/library_index - contributing + CONTRIBUTING doc_guide @@ -30,17 +30,17 @@ Indices and tables * :ref:`genindex` -.. |buildgpu| image:: https://img.shields.io/jenkins/s/http/vali.inf.ed.ac.uk/jenkins/view/marian/job/marian-dev-cuda-10.2.svg?label=CUDAC%20Build - :target: http://vali.inf.ed.ac.uk/jenkins/job/marian-dev-cuda-10.2/ - :alt: GPU build status +.. |ubuntu| image:: https://github.com/marian-nmt/marian-dev/actions/workflows/ubuntu.yml/badge.svg + :target: https://github.com/marian-nmt/marian-dev/actions/workflows/ubuntu.yml + :alt: Ubuntu build status -.. |buildcpu| image:: https://img.shields.io/jenkins/s/http/vali.inf.ed.ac.uk/jenkins/view/marian/job/marian-dev-cpu.svg?label=CPU%20Build - :target: http://vali.inf.ed.ac.uk/jenkins/job/marian-dev-cpu/ - :alt: CPU build status +.. |windows| image:: https://github.com/marian-nmt/marian-dev/actions/workflows/windows.yml/badge.svg + :target: https://github.com/marian-nmt/marian-dev/actions/workflows/windows.yml + :alt: Windows build status -.. |tests| image:: https://img.shields.io/jenkins/s/http/vali.inf.ed.ac.uk/jenkins/view/marian/job/marian-regression-tests.svg?label=Tests - :target: http://vali.inf.ed.ac.uk/jenkins/job/marian-regression-tests/ - :alt: Tests status +.. |macos| image:: https://github.com/marian-nmt/marian-dev/actions/workflows/macos.yml/badge.svg + :target: https://github.com/marian-nmt/marian-dev/actions/workflows/macos.yml + :alt: MacOS build status .. |release| image:: https://img.shields.io/github/release/marian-nmt/marian.svg?label=Release :target: https://github.com/marian-nmt/marian/releases diff --git a/doc/requirements.txt b/doc/requirements.txt index 40de5ddd..a2f87dd9 100644 --- a/doc/requirements.txt +++ b/doc/requirements.txt @@ -1,9 +1,9 @@ +lxml>=4.9.1 +docutils<=0.17 sphinx==2.4.4 breathe==4.13.0 exhale sphinx_rtd_theme myst-parser==0.14.0a3 -mistune<2.0.0 -m2r sphinx-mathjax-offline Jinja2<3.1