marty-media/server
1
0
Fork 0
Code Issues 9 Pull requests 10 Projects Releases 75 Packages Wiki Activity Actions

support S3 buckets as library location #30

Merged
martinr92 merged 69 commits from feature/s3 into develop 2023-07-01 16:27:11 +00:00
Conversation 33 Commits 69 Files changed 44 +2301 -414
martinr92 commented 2023-04-08 20:02:07 +00:00 (Migrated from gitlab.com)
Copy link

This merge request implements the AWS S3 protocol to support remote library locations.

This merge request implements the AWS S3 protocol to support remote library locations.
martinr92 commented 2023-04-08 20:02:08 +00:00 (Migrated from gitlab.com)
Copy link

assigned to @martinr92

assigned to @martinr92
martinr92 (Migrated from gitlab.com) changed target branch from main to develop 2023-04-08 20:03:37 +00:00
martinr92 commented 2023-04-08 20:12:23 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>141578cf - initial s3 implementation</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=650758533&start_sha=bf8ea8cab4a0d6af3739f1f82ecea53469737680)
martinr92 commented 2023-04-09 07:51:55 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>b10aa976 - fixed typo</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=650803720&start_sha=141578cf64aa8e33bc46bd3d0e1b2649f3426c05)
martinr92 commented 2023-04-09 08:05:31 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>489a7427 - some refactoring</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=650805007&start_sha=b10aa976e3cc807b4fe5fa7a5ee64ea7ec9a3d86)
martinr92 commented 2023-04-09 15:36:48 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

  • a2caf661 - finalized s3 list objects api

Compare with previous version

added 1 commit <ul><li>a2caf661 - finalized s3 list objects api</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=650866319&start_sha=489a742727d4c3bdb2dff14744f598fb436f8ab9)
martinr92 commented 2023-04-09 15:55:10 +00:00 (Migrated from gitlab.com)
Copy link

added 3 commits

  • e572cc6b - removed debug log
  • fa85a0d5 - new tests for library location s3
  • e80af45f - updated s3 location type api documentation

Compare with previous version

added 3 commits <ul><li>e572cc6b - removed debug log</li><li>fa85a0d5 - new tests for library location s3</li><li>e80af45f - updated s3 location type api documentation</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=650868642&start_sha=a2caf661b6f79dbc78210996ab2cd1a3bfbacfd5)
martinr92 commented 2023-04-09 16:00:54 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

  • 79c980cf - new test for invalid s3 credentials

Compare with previous version

added 1 commit <ul><li>79c980cf - new test for invalid s3 credentials</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=650869196&start_sha=e80af45f94a437ed04bbc6aa6efa223f720d356b)
martinr92 commented 2023-04-10 10:20:09 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • 6b887f5a - updated changelog for new S3 library location type
  • 685cb223 - new username and password for library location api

Compare with previous version

added 2 commits <ul><li>6b887f5a - updated changelog for new S3 library location type</li><li>685cb223 - new username and password for library location api</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=651166247&start_sha=79c980cfa38fa5e2c5c565883a2da403b8d0f17a)
martinr92 commented 2023-04-10 19:04:42 +00:00 (Migrated from gitlab.com)
Copy link

added 3 commits

  • 5dbb6611 - fixed nil-pointer exception in library location DB
  • 6e1a4ce8 - fixed api response of library location
  • b4043be6 - new validation of S3 bucket and folder of location

Compare with previous version

added 3 commits <ul><li>5dbb6611 - fixed nil-pointer exception in library location DB</li><li>6e1a4ce8 - fixed api response of library location</li><li>b4043be6 - new validation of S3 bucket and folder of location</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=651621893&start_sha=685cb2232662f9dad39e8eb29e5bd549d8abc5c7)
martinr92 commented 2023-04-29 09:41:19 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>29126f5b - new s3 head object API</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=668988984&start_sha=b4043be62752a33b74f712f1bbce9cb5bc1e7225)
martinr92 commented 2023-04-30 18:52:20 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>91cb98d2 - updated comment</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=669176552&start_sha=29126f5bbe4148a51b56af7d6decd5ba4bc4aa78)
martinr92 commented 2023-04-30 19:25:11 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>468fd828 - new s3 file download</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=669179370&start_sha=91cb98d2973959bcecd247ba5569108cfb582ae0)
martinr92 commented 2023-05-19 19:09:13 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • 6d75d1c2 - don't panic, error handling is available
  • 121ac87d - renamed folder scan to location scan

Compare with previous version

added 2 commits <ul><li>6d75d1c2 - don&#39;t panic, error handling is available</li><li>121ac87d - renamed folder scan to location scan</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=686145576&start_sha=468fd8282ca4e909e5fb32f3b1e791ceca9b2997)
martinr92 commented 2023-05-19 19:36:33 +00:00 (Migrated from gitlab.com)
Copy link

added 62 commits

  • 121ac87d...8c0c34d2 - 43 commits from branch develop
  • 8c0c34d2...e8051ada - 9 earlier commits
  • 2940ff4b - updated changelog for new S3 library location type
  • db0500c2 - new username and password for library location api
  • 8cc86708 - fixed nil-pointer exception in library location DB
  • f0093d1a - fixed api response of library location
  • c479445d - new validation of S3 bucket and folder of location
  • a5ff3e2b - new s3 head object API
  • 6dc662a9 - updated comment
  • 9900eed3 - new s3 file download
  • 197a3d6a - don't panic, error handling is available
  • 3c47df23 - renamed folder scan to location scan

Compare with previous version

added 62 commits <ul><li>121ac87d...8c0c34d2 - 43 commits from branch <code>develop</code></li><li>8c0c34d2...e8051ada - 9 earlier commits</li><li>2940ff4b - updated changelog for new S3 library location type</li><li>db0500c2 - new username and password for library location api</li><li>8cc86708 - fixed nil-pointer exception in library location DB</li><li>f0093d1a - fixed api response of library location</li><li>c479445d - new validation of S3 bucket and folder of location</li><li>a5ff3e2b - new s3 head object API</li><li>6dc662a9 - updated comment</li><li>9900eed3 - new s3 file download</li><li>197a3d6a - don&#39;t panic, error handling is available</li><li>3c47df23 - renamed folder scan to location scan</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=686161315&start_sha=121ac87dd63b7d1e4e49a65b99570ba5e952ee3a)
martinr92 commented 2023-05-19 19:38:18 +00:00 (Migrated from gitlab.com)
Copy link

added 20 commits

  • 3c47df23...6b887f5a - 10 earlier commits
  • 685cb223 - new username and password for library location api
  • 5dbb6611 - fixed nil-pointer exception in library location DB
  • 6e1a4ce8 - fixed api response of library location
  • b4043be6 - new validation of S3 bucket and folder of location
  • 29126f5b - new s3 head object API
  • 91cb98d2 - updated comment
  • 468fd828 - new s3 file download
  • 6d75d1c2 - don't panic, error handling is available
  • 121ac87d - renamed folder scan to location scan
  • 9ecd0759 - Merge branch 'feature/s3' of gitlab.com:marty-media/server into feature/s3

Compare with previous version

added 20 commits <ul><li>3c47df23...6b887f5a - 10 earlier commits</li><li>685cb223 - new username and password for library location api</li><li>5dbb6611 - fixed nil-pointer exception in library location DB</li><li>6e1a4ce8 - fixed api response of library location</li><li>b4043be6 - new validation of S3 bucket and folder of location</li><li>29126f5b - new s3 head object API</li><li>91cb98d2 - updated comment</li><li>468fd828 - new s3 file download</li><li>6d75d1c2 - don&#39;t panic, error handling is available</li><li>121ac87d - renamed folder scan to location scan</li><li>9ecd0759 - Merge branch &#39;feature/s3&#39; of gitlab.com:marty-media/server into feature/s3</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=686162432&start_sha=3c47df23dc7476752d1f04e744a582d592a9bd25)
martinr92 commented 2023-05-19 19:47:29 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

  • d165f9c2 - start api testing with clean database

Compare with previous version

added 1 commit <ul><li>d165f9c2 - start api testing with clean database</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=686167632&start_sha=9ecd07597e17d4670ce56a66b8052683e8b92b16)
martinr92 commented 2023-05-19 20:12:41 +00:00 (Migrated from gitlab.com)
Copy link

added 4 commits

  • a8c98ee6 - fail on api test call error
  • 0843b827 - enhanced error message on library creation error
  • 311db27c - de-coupling some tests
  • 79a2a954 - fixed library location test

Compare with previous version

added 4 commits <ul><li>a8c98ee6 - fail on api test call error</li><li>0843b827 - enhanced error message on library creation error</li><li>311db27c - de-coupling some tests</li><li>79a2a954 - fixed library location test</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=686181061&start_sha=d165f9c2d91feb66f5455396ba5b264bcbafabf6)
martinr92 commented 2023-05-19 20:26:15 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

  • 0c891964 - create new temporary directory for unit testing

Compare with previous version

added 1 commit <ul><li>0c891964 - create new temporary directory for unit testing</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=686188091&start_sha=79a2a954e54a4467ecd2f6bf7017a7ba94fff178)
martinr92 commented 2023-06-26 12:39:45 +00:00 (Migrated from gitlab.com)
Copy link

added 5 commits

  • ae7f336f - enhanced ffprobe log
  • 905fa779 - support limit on select
  • cbd70c0c - refactoring of scanner and analyzer jobs
  • e4936e75 - refactoring of media item update
  • bf2eed8a - new unit test for media stream update

Compare with previous version

added 5 commits <ul><li>ae7f336f - enhanced ffprobe log</li><li>905fa779 - support limit on select</li><li>cbd70c0c - refactoring of scanner and analyzer jobs</li><li>e4936e75 - refactoring of media item update</li><li>bf2eed8a - new unit test for media stream update</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=718154497&start_sha=0c89196495ce775ca94cbdbe0f8b02d568832dea)
martinr92 commented 2023-06-27 10:40:55 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • e39a0cdf - keep fields of location scan job struct private
  • ffe942b2 - refactoring for s3 scanner integration

Compare with previous version

added 2 commits <ul><li>e39a0cdf - keep fields of location scan job struct private</li><li>ffe942b2 - refactoring for s3 scanner integration</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=719203059&start_sha=bf2eed8a144dc3479ac2a1eb6d72988fc6331ca5)
martinr92 commented 2023-06-27 17:31:44 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • 19095fd9 - fixed file replacement in scanner
  • 1cc31e64 - new marking of outdated items in media scanner

Compare with previous version

added 2 commits <ul><li>19095fd9 - fixed file replacement in scanner</li><li>1cc31e64 - new marking of outdated items in media scanner</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=719748505&start_sha=ffe942b25c03b2c2f06b0754a377d1b4d958cd25)
martinr92 commented 2023-06-27 17:51:07 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • 452cb409 - removed TODO for operator flag
  • fe7a2046 - new unit test for library rescan

Compare with previous version

added 2 commits <ul><li>452cb409 - removed TODO for operator flag</li><li>fe7a2046 - new unit test for library rescan</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=719764831&start_sha=1cc31e64243c7a506ce62120861b42c22a79d8a3)
martinr92 commented 2023-06-29 15:22:20 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • 76dac4c3 - refactoring of s3 connection in API
  • 62e33094 - scan s3 remote location

Compare with previous version

added 2 commits <ul><li>76dac4c3 - refactoring of s3 connection in API</li><li>62e33094 - scan s3 remote location</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=721888518&start_sha=fe7a20467b807c29854a8116585ced96222b2b09)
martinr92 commented 2023-06-30 17:29:03 +00:00 (Migrated from gitlab.com)
Copy link

added 5 commits

  • c7831fe0 - added missing copyright
  • 3f28f57d - new version db api
  • a5a5c8be - use unix file path for s3 file download
  • ee3c6684 - new error handling during start up of backend
  • 33dd5565 - enhanced s3 error log

Compare with previous version

added 5 commits <ul><li>c7831fe0 - added missing copyright</li><li>3f28f57d - new version db api</li><li>a5a5c8be - use unix file path for s3 file download</li><li>ee3c6684 - new error handling during start up of backend</li><li>33dd5565 - enhanced s3 error log</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=723065689&start_sha=62e33094f25f673bcb21120259c80e823088a828)
martinr92 commented 2023-07-01 08:24:36 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • c0d5a5a9 - fixed s3 object calls with spaces
  • 48bf3e00 - add s3 region for unit testing

Compare with previous version

added 2 commits <ul><li>c0d5a5a9 - fixed s3 object calls with spaces</li><li>48bf3e00 - add s3 region for unit testing</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=723286035&start_sha=33dd5565c5b1926d89b225edd0e63b6701797b8a)
martinr92 commented 2023-07-01 10:00:42 +00:00 (Migrated from gitlab.com)
Copy link

added 2 commits

  • 29dd3560 - analyzer backend job supports now s3
  • b7889abf - added support for s3 region

Compare with previous version

added 2 commits <ul><li>29dd3560 - analyzer backend job supports now s3</li><li>b7889abf - added support for s3 region</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=723296559&start_sha=48bf3e00db3a2b4cbe4ac942d41cf246aac17c27)
martinr92 commented 2023-07-01 10:57:33 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

  • 5e3f630e - faster deletion detection in scanner

Compare with previous version

added 1 commit <ul><li>5e3f630e - faster deletion detection in scanner</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=723303076&start_sha=b7889abfb858643c2f510c7d7da9e165e3700c0d)
martinr92 commented 2023-07-01 16:18:41 +00:00 (Migrated from gitlab.com)
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>506777a3 - updated changelogs</li></ul> [Compare with previous version](/marty-media/server/-/merge_requests/6/diffs?diff_id=723338930&start_sha=5e3f630ef1777e04f232cea16befa00ab30f711b)
martinr92 commented 2023-07-01 16:21:40 +00:00 (Migrated from gitlab.com)
Copy link

SonarCloud Code Analysis

Quality Gate passed

Quality Gate passed

Additional information

The following metrics might not affect the Quality Gate status but improving
them will improve your project code quality.

6 Issues

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 6 Code Smells

Coverage and Duplications

40.8% 40.8% Coverage (49.1% Estimated after merge)
0.0% 0.0% Duplication (0.0% Estimated after merge)

## SonarCloud Code Analysis ## Quality Gate passed [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed.svg 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=marty-media_server&pullRequest=6) ## Additional information *The following metrics might not affect the Quality Gate status but improving* *them will improve your project code quality.* ### 6 Issues [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.svg 'Bug')](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.svg 'A')](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=BUG) [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.svg 'Vulnerability')](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.svg 'A')](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=VULNERABILITY) [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.svg 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=marty-media_server&pullRequest=6&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.svg 'A')](https://sonarcloud.io/project/security_hotspots?id=marty-media_server&pullRequest=6&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=marty-media_server&pullRequest=6&resolved=false&types=SECURITY_HOTSPOT) [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.svg 'Code Smell')](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.svg 'A')](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=CODE_SMELL) [6 Code Smells](https://sonarcloud.io/project/issues?id=marty-media_server&pullRequest=6&resolved=false&types=CODE_SMELL) ### Coverage and Duplications [![40.8%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/40.svg '40.8%')](https://sonarcloud.io/component_measures?id=marty-media_server&pullRequest=6&metric=new_coverage&view=list) [40.8% Coverage](https://sonarcloud.io/component_measures?id=marty-media_server&pullRequest=6&metric=new_coverage&view=list) (49.1% Estimated after merge) [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.svg '0.0%')](https://sonarcloud.io/component_measures?id=marty-media_server&pullRequest=6&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=marty-media_server&pullRequest=6&metric=new_duplicated_lines_density&view=list) (0.0% Estimated after merge)
martinr92 commented 2023-07-01 16:26:02 +00:00 (Migrated from gitlab.com)
Copy link

marked this merge request as ready

marked this merge request as **ready**
martinr92 commented 2023-07-01 16:27:11 +00:00 (Migrated from gitlab.com)
Copy link

mentioned in commit a12a176924

mentioned in commit a12a1769249ddcb7770526290a55814a57b7d529
martinr92 (Migrated from gitlab.com) merged commit a12a176924 into develop 2023-07-01 16:27:11 +00:00
martinr92 commented 2023-09-17 17:07:23 +00:00 (Migrated from gitlab.com)
Copy link

🎉 This MR is included in version 0.2.0-beta.1 🎉

The release is available on GitLab release.

Your semantic-release bot 📦 🚀

:tada: This MR is included in version 0.2.0-beta.1 :tada: The release is available on [GitLab release](https://gitlab.com/marty-media/server/-/releases/v0.2.0-beta.1). Your **[semantic-release](https://github.com/semantic-release/semantic-release)** bot :package: :rocket:
martinr92 commented 2023-09-18 11:59:34 +00:00 (Migrated from gitlab.com)
Copy link

🎉 This MR is included in version 0.2.0 🎉

The release is available on GitLab release.

Your semantic-release bot 📦 🚀

:tada: This MR is included in version 0.2.0 :tada: The release is available on [GitLab release](https://gitlab.com/marty-media/server/-/releases/v0.2.0). Your **[semantic-release](https://github.com/semantic-release/semantic-release)** bot :package: :rocket:
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

   
confirmed

   
critical

   
dependencies
Pull requests that update a dependency file

  
discussion

   
docker

   
documentation

   
enhancement

   
go

   
javascript

   
security
Pull requests that address a security vulnerability

  
severity:high

   
severity:low

   
severity:moderate

   
suggestion

   
support
No labels
bug
confirmed
critical
dependencies
discussion
docker
documentation
enhancement
go
javascript
security
severity:high
severity:low
severity:moderate
suggestion
support
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
marty-media/server!30
No description provided.