aboutsummaryrefslogtreecommitdiffhomepage
path: root/docs/init.html
diff options
context:
space:
mode:
Diffstat (limited to 'docs/init.html')
-rw-r--r--docs/init.html225
1 files changed, 225 insertions, 0 deletions
diff --git a/docs/init.html b/docs/init.html
new file mode 100644
index 0000000..eb5d3b3
--- /dev/null
+++ b/docs/init.html
@@ -0,0 +1,225 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta http-equiv="content-type" content="text/html;charset=utf-8">
5 <title>__init__.py</title>
6 <link rel="stylesheet" href="pycco.css">
7</head>
8<body>
9<div id='container'>
10 <div id="background"></div>
11 <div class='section'>
12 <div class='docs'><h1>__init__.py</h1></div>
13 </div>
14 <div class='clearall'>
15 <div class='section' id='section-0'>
16 <div class='docs'>
17 <div class='octowrap'>
18 <a class='octothorpe' href='#section-0'>#</a>
19 </div>
20 <p>This project syncs data from the v4 Google Sheets API.</p>
21<h1>Discovery Mode</h1>
22<p>There are a few static streams (<code>"file_metadata"</code>, <code>"spreadsheet_metadata"</code>, <code>"sheet_metadata"</code>,
23<code>"sheets_loaded"</code>) and any number of dynamic streams. There&rsquo;s one dynamic stream per sheet in the
24one Google Sheets Doc.</p>
25<h1>Sync Mode</h1>
26 </div>
27 <div class='code'>
28 <div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span>
29<span class="kn">import</span> <span class="nn">json</span>
30<span class="kn">import</span> <span class="nn">argparse</span> <span class="c1"># unused import</span>
31<span class="kn">import</span> <span class="nn">singer</span>
32<span class="kn">from</span> <span class="nn">singer</span> <span class="kn">import</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">utils</span>
33<span class="kn">from</span> <span class="nn">tap_google_sheets.client</span> <span class="kn">import</span> <span class="n">GoogleClient</span>
34<span class="kn">from</span> <span class="nn">tap_google_sheets.discover</span> <span class="kn">import</span> <span class="n">discover</span>
35<span class="kn">from</span> <span class="nn">tap_google_sheets.sync</span> <span class="kn">import</span> <span class="n">sync</span>
36
37<span class="n">LOGGER</span> <span class="o">=</span> <span class="n">singer</span><span class="o">.</span><span class="n">get_logger</span><span class="p">()</span></pre></div>
38 </div>
39 </div>
40 <div class='clearall'></div>
41 <div class='section' id='section-1'>
42 <div class='docs'>
43 <div class='octowrap'>
44 <a class='octothorpe' href='#section-1'>#</a>
45 </div>
46 <h1>Configuration</h1>
47 </div>
48 <div class='code'>
49 <div class="highlight"><pre></pre></div>
50 </div>
51 </div>
52 <div class='clearall'></div>
53 <div class='section' id='section-2'>
54 <div class='docs'>
55 <div class='octowrap'>
56 <a class='octothorpe' href='#section-2'>#</a>
57 </div>
58 <p>This is a typical OAuth2 tap. So in a config file we expect the following keys.</p>
59<ul>
60<li>
61<p>OAuth Related:</p>
62<ul>
63<li><code>client_id</code></li>
64<li><code>client_secret</code></li>
65<li><code>refresh_token</code></li>
66</ul>
67</li>
68<li>
69<p>Tap related:</p>
70<ul>
71<li><code>spreadsheet_id</code></li>
72<li><code>start_date</code></li>
73<li><code>user_agent</code></li>
74</ul>
75</li>
76</ul>
77 </div>
78 <div class='code'>
79 <div class="highlight"><pre><span class="n">REQUIRED_CONFIG_KEYS</span> <span class="o">=</span> <span class="p">[</span>
80 <span class="s1">&#39;client_id&#39;</span><span class="p">,</span>
81 <span class="s1">&#39;client_secret&#39;</span><span class="p">,</span>
82 <span class="s1">&#39;refresh_token&#39;</span><span class="p">,</span>
83 <span class="s1">&#39;spreadsheet_id&#39;</span><span class="p">,</span>
84 <span class="s1">&#39;start_date&#39;</span><span class="p">,</span>
85 <span class="s1">&#39;user_agent&#39;</span>
86<span class="p">]</span></pre></div>
87 </div>
88 </div>
89 <div class='clearall'></div>
90 <div class='section' id='section-3'>
91 <div class='docs'>
92 <div class='octowrap'>
93 <a class='octothorpe' href='#section-3'>#</a>
94 </div>
95 <h1>Discovery Mode</h1>
96 </div>
97 <div class='code'>
98 <div class="highlight"><pre></pre></div>
99 </div>
100 </div>
101 <div class='clearall'></div>
102 <div class='section' id='section-4'>
103 <div class='docs'>
104 <div class='octowrap'>
105 <a class='octothorpe' href='#section-4'>#</a>
106 </div>
107 <p>Creates a Singer Catalog and writes it to STDOUT</p>
108 </div>
109 <div class='code'>
110 <div class="highlight"><pre><span class="k">def</span> <span class="nf">do_discover</span><span class="p">(</span><span class="n">client</span><span class="p">,</span> <span class="n">spreadsheet_id</span><span class="p">):</span></pre></div>
111 </div>
112 </div>
113 <div class='clearall'></div>
114 <div class='section' id='section-5'>
115 <div class='docs'>
116 <div class='octowrap'>
117 <a class='octothorpe' href='#section-5'>#</a>
118 </div>
119 <p>Inputs:</p>
120<ul>
121<li><code>client</code></li>
122<li>An instance of the GoogleClient class</li>
123<li><code>spreadsheet_id</code></li>
124<li>The id of the Google Sheet</li>
125</ul>
126<p>Returns:</p>
127<ul>
128<li>None</li>
129</ul>
130<p>Side Effects:</p>
131<ul>
132<li>Writes to STDOUT</li>
133</ul>
134 </div>
135 <div class='code'>
136 <div class="highlight"><pre> <span class="n">LOGGER</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">&#39;Starting discover&#39;</span><span class="p">)</span>
137 <span class="n">catalog</span> <span class="o">=</span> <span class="n">discover</span><span class="p">(</span><span class="n">client</span><span class="p">,</span> <span class="n">spreadsheet_id</span><span class="p">)</span>
138 <span class="n">json</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">catalog</span><span class="o">.</span><span class="n">to_dict</span><span class="p">(),</span> <span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
139 <span class="n">LOGGER</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">&#39;Finished discover&#39;</span><span class="p">)</span></pre></div>
140 </div>
141 </div>
142 <div class='clearall'></div>
143 <div class='section' id='section-6'>
144 <div class='docs'>
145 <div class='octowrap'>
146 <a class='octothorpe' href='#section-6'>#</a>
147 </div>
148 <h1>Entrypoint</h1>
149 </div>
150 <div class='code'>
151 <div class="highlight"><pre></pre></div>
152 </div>
153 </div>
154 <div class='clearall'></div>
155 <div class='section' id='section-7'>
156 <div class='docs'>
157 <div class='octowrap'>
158 <a class='octothorpe' href='#section-7'>#</a>
159 </div>
160 <p>Read a config, then run discovery mode or sync mode</p>
161 </div>
162 <div class='code'>
163 <div class="highlight"><pre><span class="nd">@singer</span><span class="o">.</span><span class="n">utils</span><span class="o">.</span><span class="n">handle_top_exception</span><span class="p">(</span><span class="n">LOGGER</span><span class="p">)</span>
164<span class="k">def</span> <span class="nf">main</span><span class="p">():</span></pre></div>
165 </div>
166 </div>
167 <div class='clearall'></div>
168 <div class='section' id='section-8'>
169 <div class='docs'>
170 <div class='octowrap'>
171 <a class='octothorpe' href='#section-8'>#</a>
172 </div>
173 <p>Inputs:</p>
174<ul>
175<li>None</li>
176</ul>
177<p>Returns:</p>
178<ul>
179<li>None</li>
180</ul>
181<p>Side Effects:</p>
182<ul>
183<li>Writes to STDOUT</li>
184</ul>
185 </div>
186 <div class='code'>
187 <div class="highlight"><pre> <span class="n">parsed_args</span> <span class="o">=</span> <span class="n">singer</span><span class="o">.</span><span class="n">utils</span><span class="o">.</span><span class="n">parse_args</span><span class="p">(</span><span class="n">REQUIRED_CONFIG_KEYS</span><span class="p">)</span>
188
189 <span class="k">with</span> <span class="n">GoogleClient</span><span class="p">(</span><span class="n">parsed_args</span><span class="o">.</span><span class="n">config</span><span class="p">[</span><span class="s1">&#39;access_token&#39;</span><span class="p">],</span>
190 <span class="n">parsed_args</span><span class="o">.</span><span class="n">config</span><span class="p">[</span><span class="s1">&#39;user_agent&#39;</span><span class="p">])</span> <span class="k">as</span> <span class="n">client</span><span class="p">:</span>
191
192 <span class="n">state</span> <span class="o">=</span> <span class="p">{}</span>
193 <span class="k">if</span> <span class="n">parsed_args</span><span class="o">.</span><span class="n">state</span><span class="p">:</span>
194 <span class="n">state</span> <span class="o">=</span> <span class="n">parsed_args</span><span class="o">.</span><span class="n">state</span>
195
196 <span class="n">config</span> <span class="o">=</span> <span class="n">parsed_args</span><span class="o">.</span><span class="n">config</span>
197 <span class="n">spreadsheet_id</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;spreadsheet_id&#39;</span><span class="p">)</span>
198
199 <span class="k">if</span> <span class="n">parsed_args</span><span class="o">.</span><span class="n">discover</span><span class="p">:</span>
200 <span class="n">do_discover</span><span class="p">(</span><span class="n">client</span><span class="p">,</span> <span class="n">spreadsheet_id</span><span class="p">)</span>
201 <span class="k">elif</span> <span class="n">parsed_args</span><span class="o">.</span><span class="n">catalog</span><span class="p">:</span>
202 <span class="n">sync</span><span class="p">(</span><span class="n">client</span><span class="o">=</span><span class="n">client</span><span class="p">,</span>
203 <span class="n">config</span><span class="o">=</span><span class="n">config</span><span class="p">,</span>
204 <span class="n">catalog</span><span class="o">=</span><span class="n">parsed_args</span><span class="o">.</span><span class="n">catalog</span><span class="p">,</span>
205 <span class="n">state</span><span class="o">=</span><span class="n">state</span><span class="p">)</span></pre></div>
206 </div>
207 </div>
208 <div class='clearall'></div>
209 <div class='section' id='section-9'>
210 <div class='docs'>
211 <div class='octowrap'>
212 <a class='octothorpe' href='#section-9'>#</a>
213 </div>
214 <p>Unused</p>
215 </div>
216 <div class='code'>
217 <div class="highlight"><pre><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
218 <span class="n">main</span><span class="p">()</span>
219
220</pre></div>
221 </div>
222 </div>
223 <div class='clearall'></div>
224</div>
225</body>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-21 22:07:41 +0000