Updating YouTube videos via the YouTube Data API using Emacs Lisp and url-http-oauth
| elisp, emacs, emacsconf, youtube, videoWe upload EmacsConf videos to both YouTube and Toobnix, which is a PeerTube instance. This makes it easier for people to come across them after the conference.
I can upload to Toobnix and set titles and descriptions using the peertube-cli tool. I tried a Python script for uploading to YouTube, but it was a bit annoying due to quota restrictions. Instead, I uploaded the videos by dragging and dropping them into YouTube Studio. This allowed me to upload 15 at a time.
The videos on YouTube had just the filenames. I wanted to rename the videos and set the descriptions. In 2022, I used xdotool, simulating mouse clicks and pasting in text for larger text blocks.
Xdotool script
(defun my-xdotool-insert-mouse-location (interactive) (let ((pos (shell-command-to-string "xdotool getmouselocation"))) (when (string-match "x:\\([0-9]+\\) y:\\([0-9]+\\)" pos) (insert (format "(shell-command \"xdotool mousemove %s %s click 1\")\n" (match-string 1 pos) (match-string 2 pos)))))) (setq list (seq-filter (lambda (o) (and (file-exists-p (expand-file-name (concat (plist-get o :video-slug) "--final.webm") emacsconf-cache-dir)) (null (plist-get o :youtube-url)))) (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))) (while list (progn (shell-command "xdotool mousemove 707 812 click 1 sleep 2") (setq talk (pop list)) ;; click create (shell-command "xdotool mousemove 843 187 click 1 sleep 1") ;; video (shell-command "xdotool mousemove 833 217 click 1 sleep 1") ;; select files (shell-command (concat "xdotool mousemove 491 760 click 1 sleep 4 type " (shell-quote-argument (concat (plist-get talk :video-slug) "--final.webm")))) ;; open (shell-command "xdotool mousemove 1318 847 click 1 sleep 5") (kill-new (concat emacsconf-name " " emacsconf-year ": " (plist-get talk :title) " - " (plist-get talk :speakers-with-pronouns))) (shell-command "xdotool sleep 1 mousemove 331 440 click :1 key Ctrl+a Delete sleep 1 key Ctrl+Shift+v sleep 2") (kill-new (emacsconf-publish-video-description talk t)) (shell-command "xdotool mousemove 474 632 click 1 sleep 1 key Ctrl+a sleep 1 key Delete sleep 1 key Ctrl+Shift+v")) (read-string "Press a key once you've pasted in the description") ;; next (when (emacsconf-captions-edited-p (expand-file-name (concat (plist-get talk :video-slug) "--main.vtt") emacsconf-cache-dir)) (shell-command "xdotool mousemove 352 285 click 1 sleep 1") ;; add captions (shell-command "xdotool mousemove 877 474 click 1 sleep 3") (shell-command "xdotool mousemove 165 408 click 1 sleep 1") (shell-command "xdotool mousemove 633 740 click 1 sleep 2") (shell-command (concat "xdotool mousemove 914 755 click 1 sleep 4 type " (shell-quote-argument (concat (plist-get talk :video-slug) "--main.vtt")))) (read-string "Press a key once you've loaded the VTT") (shell-command "xdotool mousemove 910 1037 sleep 1 click 1 sleep 4") ;; done (shell-command "xdotool mousemove 890 297 click 1 sleep 3") ) (progn ;; visibility (shell-command "xdotool mousemove 810 303 click 1 sleep 2") ;; public (shell-command "xdotool mousemove 119 614 click 1 sleep 2") ;; copy (shell-command "xdotool mousemove 882 669 click 1 sleep 1") ;; done (shell-command "xdotool mousemove 908 1089 click 1 sleep 5 key Alt+Tab") (emacsconf-with-talk-heading talk (org-entry-put (point) "YOUTUBE_URL" (read-string "URL: ")) )) )
Using xdotool wasn't very elegant, since I needed to figure out the coordinates for each click. I tried using Spookfox to control Mozilla Firefox from Emacs, but Youtube's editing interface didn't seem to have any textboxes that I could set. I decided to use EmacsConf 2023 as an excuse to learn how to talk to the Youtube Data API, which required figuring out OAuth. Even though it was easy to find examples in Python and NodeJS, I wanted to see if I could stick with using Emacs Lisp so that I could add the code to the emacsconf-el repository.
After a quick search, I picked url-http-oauth as the library that I'd
try first. I used the url-http-oauth-demo.el included in the package
to figure out what to set for the YouTube Data API. I wrote a function
to make getting the redirect URL easier
(emacsconf-extract-oauth-browse-and-prompt). Once I authenticated
successfully, I explored using alphapapa's plz library. It can handle
finding the JSON object and parsing it out for me. With it, I updated
videos to include titles and descriptions from my Emacs code, and I
copied the video IDs into my Org properties.
emacsconf-extract.el code for Youtube renaming
;;; YouTube ;; When the token needs refreshing, delete the associated lines from ;; ~/.authinfo This code just sets the title and description. Still ;; need to figure out how to properly set the license, visibility, ;; recording date, and captions. ;; ;; To avoid being prompted for the client secret, it's helpful to have a line in ~/.authinfo or ~/.authinfo.gpg with ;; machine https://oauth2.googleapis.com/token username CLIENT_ID password CLIENT_SECRET (defvar emacsconf-extract-google-client-identifier nil) (defvar emacsconf-extract-youtube-api-channels nil) (defvar emacsconf-extract-youtube-api-categories nil) (defun emacsconf-extract-oauth-browse-and-prompt (url) "Open URL and wait for the redirected code URL." (browse-url url) (read-from-minibuffer "Paste the redirected code URL: ")) (defun emacsconf-extract-youtube-api-setup () (interactive) (require 'plz) (require 'url-http-oauth) (when (getenv "GOOGLE_APPLICATION_CREDENTIALS") (let-alist (json-read-file (getenv "GOOGLE_APPLICATION_CREDENTIALS")) (setq emacsconf-extract-google-client-identifier .web.client_id))) (unless (url-http-oauth-interposed-p "https://youtube.googleapis.com/youtube/v3/") (url-http-oauth-interpose `(("client-identifier" . ,emacsconf-extract-google-client-identifier) ("resource-url" . "https://youtube.googleapis.com/youtube/v3/") ("authorization-code-function" . emacsconf-extract-oauth-browse-and-prompt) ("authorization-endpoint" . "https://accounts.google.com/o/oauth2/v2/auth") ("authorization-extra-arguments" . (("redirect_uri" . "http://localhost:8080"))) ("access-token-endpoint" . "https://oauth2.googleapis.com/token") ("scope" . "https://www.googleapis.com/auth/youtube") ("client-secret-method" . prompt)))) (setq emacsconf-extract-youtube-api-channels (plz 'get "https://youtube.googleapis.com/youtube/v3/channels?part=contentDetails&mine=true" :headers `(("Authorization" . ,(url-oauth-auth "https://youtube.googleapis.com/youtube/v3/"))) :as #'json-read)) (setq emacsconf-extract-youtube-api-categories (plz 'get "https://youtube.googleapis.com/youtube/v3/videoCategories?part=snippet®ionCode=CA" :headers `(("Authorization" . ,(url-oauth-auth "https://youtube.googleapis.com/youtube/v3/"))) :as #'json-read)) (setq emacsconf-extract-youtube-api-videos (plz 'get (concat "https://youtube.googleapis.com/youtube/v3/playlistItems?part=snippet,contentDetails,status&forMine=true&order=date&maxResults=50&playlistId=" (url-hexify-string (let-alist (elt (assoc-default 'items emacsconf-extract-youtube-api-channels) 0) .contentDetails.relatedPlaylists.uploads) )) :headers `(("Authorization" . ,(url-oauth-auth "https://youtube.googleapis.com/youtube/v3/"))) :as #'json-read))) (defvar emacsconf-extract-youtube-tags '("emacs" "emacsconf")) (defun emacsconf-extract-youtube-object (video-id talk &optional privacy-status) "Format the video object for VIDEO-ID using TALK details." (setq privacy-status (or privacy-status "unlisted")) (let ((properties (emacsconf-publish-talk-video-properties talk 'youtube))) `((id . ,video-id) (kind . "youtube#video") (snippet (categoryId . "28") (title . ,(plist-get properties :title)) (tags . ,emacsconf-extract-youtube-tags) (description . ,(plist-get properties :description)) ;; Even though I set recordingDetails and status, it doesn't seem to stick. ;; I'll leave this in here in case someone else can figure it out. (recordingDetails (recordingDate . ,(format-time-string "%Y-%m-%dT%TZ" (plist-get talk :start-time) t)))) (status (privacyStatus . "unlisted") (license . "creativeCommon"))))) (defun emacsconf-extract-youtube-api-update-video (video-object) "Update VIDEO-OBJECT." (let-alist video-object (let* ((slug (cond ;; not yet renamed ((string-match (rx (literal emacsconf-id) " " (literal emacsconf-year) " " (group (1+ (or (syntax word) "-"))) " ") .snippet.title) (match-string 1 .snippet.title)) ;; renamed, match the description instead ((string-match (rx (literal emacsconf-base-url) (literal emacsconf-year) "/talks/" (group (1+ (or (syntax word) "-")))) .snippet.description) (match-string 1 .snippet.description)) ;; can't find, prompt (t (when (string-match (rx (literal emacsconf-id) " " (literal emacsconf-year)) .snippet.title) (completing-read (format "Slug for %s: " .snippet.title) (seq-map (lambda (o) (plist-get o :slug)) (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))))))) (video-id .snippet.resourceId.videoId) (id .id) result) (when slug ;; set the YOUTUBE_URL property (emacsconf-with-talk-heading slug (org-entry-put (point) "YOUTUBE_URL" (concat "https://www.youtube.com/watch?v=" video-id)) (org-entry-put (point) "YOUTUBE_ID" id)) (plz 'put "https://www.googleapis.com/youtube/v3/videos?part=snippet,recordingDetails,status" :headers `(("Authorization" . ,(url-oauth-auth "https://youtube.googleapis.com/youtube/v3/")) ("Accept" . "application/json") ("Content-Type" . "application/json")) :body (json-encode (emacsconf-extract-youtube-object video-id (emacsconf-resolve-talk slug)))))))) (defun emacsconf-extract-youtube-rename-videos (&optional videos) "Rename videos and set the YOUTUBE_URL property in the Org heading." (let ((info (emacsconf-get-talk-info))) (mapc (lambda (video) (when (string-match (rx (literal emacsconf-id) " " (literal emacsconf-year))) (emacsconf-extract-youtube-api-update-video video))) (assoc-default 'items (or videos emacsconf-extract-youtube-api-videos))))) (provide 'emacsconf-extract)
I haven't quite figured out how to set status and recordingDetails
properly. The code sets them, but they don't stick. That's okay. I
think I can set those as a batch operation. It looks like I need to
change visibility one by one, though, which might be a good
opportunity to check the end of the video for anything that needs to
be trimmed off.
I also want to figure out how to upload captions. I'm not entirely
sure how to do multipart form data yet with the url library or
plz. It might be nice to someday set up an HTTP server so that Emacs
can handle OAuth redirects itself. I'll save that for another blog
post and share my notes for now.
This code is in emacsconf-extract.el.