[ckan-changes] [ckan/ckan] 7dfd08: Storing the size and mimetypes from uploads

Wed Oct 26 10:00:30 UTC 2016

  Branch: refs/heads/master
  Home:   https://github.com/ckan/ckan
  Commit: 7dfd08c0866a2e5cf1b765e0ec1546447380fb4e
      https://github.com/ckan/ckan/commit/7dfd08c0866a2e5cf1b765e0ec1546447380fb4e
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py
    M requirements.in

  Log Message:
  -----------
  Storing the size and mimetypes from uploads

size is stored as bytes

there's a bit of redundancy with getting the mimetypes of files, by
default, mimetype is determined by the url, fallback is done by using
`python-magic` module

  Commit: 90ed8f76a956deed8a041bdeab6495b13893d873
      https://github.com/ckan/ckan/commit/90ed8f76a956deed8a041bdeab6495b13893d873
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/uploader.py

  Log Message:
  -----------
  Specified which exception to catch for python-magic call failure

  Commit: 38681d6b6f6775b62d188447a79c60f29a55577e
      https://github.com/ckan/ckan/commit/38681d6b6f6775b62d188447a79c60f29a55577e
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py

  Log Message:
  -----------
  Dont override size and mimetype if specified by the user

  Commit: 3dc9ed9811da79767ad1644b10ec8cca2a0e58e4
      https://github.com/ckan/ckan/commit/3dc9ed9811da79767ad1644b10ec8cca2a0e58e4
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py

  Log Message:
  -----------
  Fixing code placement due to failing many tests, moved filesize and mimetype in ResourceUpload init method instead of upload method

  Commit: 8d80e366f00b569a64961d1f0807c2f91554ba8b
      https://github.com/ckan/ckan/commit/8d80e366f00b569a64961d1f0807c2f91554ba8b
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/tests/logic/action/test_create.py

  Log Message:
  -----------
  Added some test for mimetype and size

Testing by user specified parameters are passing,
not sure how to approach testing real uploads in the tests

  Commit: f75cffa76963b603481651410637599b31d6e35a
      https://github.com/ckan/ckan/commit/f75cffa76963b603481651410637599b31d6e35a
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/tests/logic/action/test_create.py

  Log Message:
  -----------
  Basic tests in place, need a better way to providing a file for real upload to the resource_create action

  Commit: 96053f2527f63aad95efddc91df815c094aa9893
      https://github.com/ckan/ckan/commit/96053f2527f63aad95efddc91df815c094aa9893
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py
    M ckan/tests/lib/dictization/test_model_dictize.py
    M ckan/tests/logic/action/test_create.py
    M ckan/tests/logic/action/test_update.py

  Log Message:
  -----------
  Finishing test_create tests, moved mimetype logic back into uploader.py,
test_model_dictize.py had one test that now expects size to be 0, fixing
formatting for pip8

  Commit: 7cfaaa17696e25e58383764603af80c57ffcac22
      https://github.com/ckan/ckan/commit/7cfaaa17696e25e58383764603af80c57ffcac22
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/logic/action/update.py

  Log Message:
  -----------
  updating resources that dont have an upload will not update the size var

  Commit: 64d55e140ffca72eaf0715d2a643dd270f8e71f7
      https://github.com/ckan/ckan/commit/64d55e140ffca72eaf0715d2a643dd270f8e71f7
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/tests/logic/action/test_update.py

  Log Message:
  -----------
  Added tests for updating a resource

The included tests have a similar process as the tests for creating a
resource.

  Commit: 00738b59bc5d0b3b9164c7f85c0ec88154ad0b36
      https://github.com/ckan/ckan/commit/00738b59bc5d0b3b9164c7f85c0ec88154ad0b36
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M requirements.in
    M requirements.txt

  Log Message:
  -----------
  Pinning python-magic to 0.4.6 and adding to requirements.txt

  Commit: 0764a3c2163faa2a6da6b6f25416037a0db36031
      https://github.com/ckan/ckan/commit/0764a3c2163faa2a6da6b6f25416037a0db36031
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/uploader.py

  Log Message:
  -----------
  Added config option for guessing mimetype

  Commit: e42cf00dfc4f9e74aac58bbd707842014281aae2
      https://github.com/ckan/ckan/commit/e42cf00dfc4f9e74aac58bbd707842014281aae2
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M test-core.ini

  Log Message:
  -----------
  Adding resource upload directory for circleci

  Commit: 533071dbbe4221f4b193977b18574873366c4f86
      https://github.com/ckan/ckan/commit/533071dbbe4221f4b193977b18574873366c4f86
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/tests/logic/action/test_create.py
    M ckan/tests/logic/action/test_update.py

  Log Message:
  -----------
  Could have saved myself many hours if i knew @helpers.change_config was a thing

  Commit: 8ad4c26564b490db682180b475362540bb5cb4fa
      https://github.com/ckan/ckan/commit/8ad4c26564b490db682180b475362540bb5cb4fa
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M doc/maintaining/configuration.rst

  Log Message:
  -----------
  Added config doc info for ckan.mimetype_guess

  Commit: 8c5298dc701b41b43b5ee0c6e43be0a3f5d5a58b
      https://github.com/ckan/ckan/commit/8c5298dc701b41b43b5ee0c6e43be0a3f5d5a58b
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/tests/logic/action/test_update.py

  Log Message:
  -----------
  Fixing pep8

Remove whitespace

  Commit: 0045b8476036f645483cc5a73f93ce00401af526
      https://github.com/ckan/ckan/commit/0045b8476036f645483cc5a73f93ce00401af526
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/logic/action/update.py

  Log Message:
  -----------
  Reverting changes on Upload class in uploader

  Commit: d381d1e3a3c5bb28f3c183e7b24c191272312572
      https://github.com/ckan/ckan/commit/d381d1e3a3c5bb28f3c183e7b24c191272312572
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M test-core.ini

  Log Message:
  -----------
  Disable upload storage, causing webtest bug for file uploads

  Commit: 88531cccf3ec669fb657ff8d6cddcf85ce57d517
      https://github.com/ckan/ckan/commit/88531cccf3ec669fb657ff8d6cddcf85ce57d517
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/tests/logic/action/test_create.py
    M ckan/tests/logic/action/test_update.py

  Log Message:
  -----------
  @amercader is a hero, test passing with fakefs
Fixing pep8

  Commit: 86b4e4a2340a5c6db6b4a19eb7653741c832936f
      https://github.com/ckan/ckan/commit/86b4e4a2340a5c6db6b4a19eb7653741c832936f
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/tests/logic/action/test_create.py
    M ckan/tests/logic/action/test_update.py

  Log Message:
  -----------
  Fixing docstrings to use ''' instead of """

In accordance to the coding standards

  Commit: bf90c31ddb175dbe2219225b6c5d0559e6ae063e
      https://github.com/ckan/ckan/commit/bf90c31ddb175dbe2219225b6c5d0559e6ae063e
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/dictization/__init__.py
    M ckan/model/resource.py

  Log Message:
  -----------
  Adding long data type to table_dictize, defaulting size column to be 0

  Commit: 69bf8c6a69b081ef38f075942a9dd85c713995ec
      https://github.com/ckan/ckan/commit/69bf8c6a69b081ef38f075942a9dd85c713995ec
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/model/resource.py
    M ckan/tests/lib/dictization/test_model_dictize.py

  Log Message:
  -----------
  Reverting size initialization, have size value only when there's a file upload

  Commit: 8be9d720807ecfc467b06644c80f7d795ba809b4
      https://github.com/ckan/ckan/commit/8be9d720807ecfc467b06644c80f7d795ba809b4
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py

  Log Message:
  -----------
  Pep8 var comparison

  Commit: 50c66b6822b1a18d523a9afa816b421fae31ab01
      https://github.com/ckan/ckan/commit/50c66b6822b1a18d523a9afa816b421fae31ab01
  Author: Jared Smith <jared.smith.jrod at gmail.com>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M CHANGELOG.rst

  Log Message:
  -----------
  Note for size type change

  Commit: 9de1f467d852d2fac75217cfd36b400d779e718e
      https://github.com/ckan/ckan/commit/9de1f467d852d2fac75217cfd36b400d779e718e
  Author: Adrià Mercader <amercadero at gmail.com>
  Date:   2016-10-26 (Wed, 26 Oct 2016)

  Changed paths:
    M CHANGELOG.rst
    M ckan/lib/dictization/__init__.py
    M ckan/lib/uploader.py
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py
    M ckan/tests/logic/action/test_create.py
    M ckan/tests/logic/action/test_update.py
    M doc/maintaining/configuration.rst
    M requirements.in
    M requirements.txt

  Log Message:
  -----------
  Merge pull request #3205 from HighwayThree/issue_1517_size_mimetypes

Storing the size and mimetypes from uploads

Compare: https://github.com/ckan/ckan/compare/c0d88b56d954...9de1f467d852