Commit Graph

200 Commits

Author SHA1 Message Date
espie
935e9f2b7c cleaner way to deal with paths, no functional change 2019-07-21 11:10:23 +00:00
espie
70d0d70b25 fix sqlports, bump to make sure 2019-07-20 23:12:47 +00:00
sthen
98e1189996 Add a belt-and-braces check for the number of lines output in the produced
ports-INDEX, error out if it's too small.
2019-07-18 06:35:43 +00:00
espie
d45c8394c9 bye bye old PERMIT_* 2019-07-14 11:27:19 +00:00
sthen
9fe1e38b23 replace simple PERMIT_PACKAGE_CDROM=Yes with PERMIT_PACKAGE=Yes 2019-07-12 20:43:27 +00:00
espie
3ec73388de make sure to stop if scripts fail
as noticed by sthen@
2019-06-04 16:05:52 +00:00
espie
7271781869 phase out old PERMIT_* variables.
We won't be distributing CDROM anymore, so simplify to
just PERMIT_PACKAGE / PERMIT_DISTFILES

In particular, the new variables are shorter, so this makes
for better cosmetic sense in ports.

The "current" version allows for the old variables to die out
peacefully, at our leasure, and then I'll remove the old stubs.

As discussed with sthen@, various people agree, and deraadt@
is okay with the strategic change.

This does survive a bulk.

PLEASE NOTE: built packages require current pkg* tools.
In particular, the current version deals with
@comment pkgpath=* ftp=*
just fine, but the old one WILL COMPLAIN about missing cdrom info.

register-plist also doesn't care, adding/removing cdrom info
is a no-op for it. There is NO BUMP needed for the conversion.

For now, sqlports STILL carries the old variables. I'll deal
with their removal later.
2019-05-31 21:27:03 +00:00
espie
71a2ef4f62 Add a denormalized table that instantiates the main ports view.
So, Portsq is a snapshot of Ports... dirty but fast.

I had mixed feelings about this.
There's also a script to resync the table.

Grows the db by about 50% (+25MB)

Discussed with sthen@
2019-05-18 21:35:21 +00:00
espie
78fb9be676 create indices for all fullpkgpaths in secondary tables, this ought
to speed up some requests at least a little bit.
2019-05-17 20:41:54 +00:00
espie
9f70fcb4b8 Fix bug in argument order, strenghten argument parsing to avoid that.
Add a "meta" table that just contains a schema version and a hash.

So that some clients (e.g., portroach) can automatically figure out
whether a rebuild is required.
2019-03-15 11:29:53 +00:00
espie
86c6128347 slightly more correct subrequest (fullpkgpath is ordered "by accident")
I suspected this, confirmed after a discussion with Reda Dehak, from my
lab at work.
2019-03-11 13:29:21 +00:00
espie
4c1a6388c5 use "chained joins" to create canonical_depends with the Sql.pm framework
kill a bit of code.

adjust is now the only request "not in the mold", so just create it when
needed (so, late enough)

Add index creation (directly in create_schema), for now used for
canonical.
2019-01-22 16:55:22 +00:00
espie
f70efd5390 rename a few methods and finish implementing chaining joins 2019-01-21 08:51:56 +00:00
espie
3e31d033e5 somewhat more extensive diagnostics 2019-01-17 17:39:00 +00:00
espie
037e1af327 better error reporting 2019-01-14 18:38:04 +00:00
espie
1f4ec730cf allow us to ask for indexed/non indexed variables 2019-01-14 18:37:15 +00:00
espie
b44acbd441 use (SELECT ) inside INSERT for keyword tables, so that less actual perl
code is needed.

no actual schema change
2019-01-12 13:57:41 +00:00
espie
8a522c95ec incorporate shared_libs into main view 2019-01-12 11:10:00 +00:00
espie
98537e4d7f put distfiles in its own table along with indexers 2019-01-11 21:52:25 +00:00
espie
0af4f8081b fix comma location with columns with an expr 2019-01-11 21:22:26 +00:00
espie
d906df9046 fix wantlib view to not create bogus entries
create a wantlib_ordered view with full lists of wantlib
re-incorporate wantlib into main ports view
2019-01-11 19:52:14 +00:00
espie
3196ba8cf0 stop inheriting from Select, but forward appropriate methods instead 2019-01-11 19:51:39 +00:00
espie
e14a11decb fix the _wantlib table (it should separate >=10 as extra)
AND amend the wantlib view so that it has the "complete" wantlib
2019-01-11 17:17:13 +00:00
espie
a60e3c3634 make the View::Column display smarter: intuit join/base table names
(default to join for obvious reasons)
2019-01-11 15:44:59 +00:00
espie
388141c83e rewrite ::Expr to be more sensible wrt complicated expressions
ditch the "origin/origin_name" nonsense
2019-01-11 10:26:51 +00:00
espie
00cb60d1c9 todo 2019-01-10 13:44:22 +00:00
espie
ee84de609c add descr/readme to ports view, simplify print-ports-index
don't left join on categories, it's always here.

make 'alter-tables' full

wantlib_ordered does not work yet... more stuff to look at.
2019-01-09 15:46:20 +00:00
espie
0bb25887ea fix table name in canonical_depends, somehow my tests missed it :(
Re-add EqualConstant, so that I can actually error out if names are NOT
there.
2019-01-09 12:59:44 +00:00
espie
8c318cc6cd simplify table name handling 2019-01-08 23:28:15 +00:00
espie
2a18423aaf no longer needed 2019-01-08 19:43:14 +00:00
espie
ae88700f3a move to using Sql for most tables and Views, no functional change.
The main benefit is that it's easier to construct the schema from several
places, with proper indentation.

Also, sql knows a bit about sql, so it's able to
- know which column is a foreign key, just reference the table
- remove table aliases when they're not needed
- leave table names out in table.column when there's no ambiguity.

For instance, view Ports now looks like:
CREATE VIEW Ports AS
    SELECT
        Id AS PathId,
        _Paths.FullPkgPath AS FullPkgPath,
        _AutoVersion.Value AS AUTOCONF_VERSION,
        T0001.Value AS AUTOMAKE_VERSION,
[...]
    FROM _Ports
        JOIN _Paths
            ON Canonical=_Ports.FullPkgPath
        LEFT JOIN _AutoVersion
            ON _AutoVersion.KeyRef=AUTOCONF_VERSION
        LEFT JOIN _AutoVersion T0001
            ON T0001.KeyRef=AUTOMAKE_VERSION


instead of:
CREATE VIEW Ports AS
    SELECT
        T0057.Id AS PathId,
        T0057.FULLPKGPATH AS FULLPKGPATH,
        T0058.VALUE AS AUTOCONF_VERSION,
        T0059.VALUE AS AUTOMAKE_VERSION,
[...]
    FROM _Ports
        JOIN _Paths T0057
            ON  T0057.Canonical=_Ports.FULLPKGPATH
        LEFT JOIN _AutoVersion T0058
            ON  T0058.KEYREF=_Ports.AUTOCONF_VERSION
        LEFT JOIN _AutoVersion T0059
            ON  T0059.KEYREF=_Ports.AUTOMAKE_VERSION
2019-01-08 19:42:45 +00:00
espie
b219dba076 trim table names in on part as well if we can
sort table/views for easier comparison
have table aliases per-select
2019-01-07 06:09:40 +00:00
espie
d3ed04ee84 have prepend respect the order of added columns
add Sql::IsNull join constraint
2019-01-02 22:49:42 +00:00
espie
b8657e8faf fix prepend for views (need to go to the select, like add)
and add a "constant" join, because ordered_depends.2  does not mean much...
2019-01-02 15:37:51 +00:00
espie
95a54c499a column_names + better diagnostics for incomplete reference. 2019-01-01 16:23:10 +00:00
espie
30bc3e29d9 ignore spaces around () 2018-12-30 19:51:38 +00:00
espie
e77af40487 typo 2018-12-30 19:08:25 +00:00
espie
63e702113d have the inserter order match the table order, much less confusing for later 2018-12-30 10:40:32 +00:00
espie
89aa414f28 allow multi-line stringize (for expr) and indent each accordingly
allow view columns to double as group-by entries, because it's often
the case
2018-12-28 10:48:57 +00:00
espie
8a87cd302c references can know which field it's referring to (table key) in many cases 2018-12-27 12:37:33 +00:00
espie
d34a457a74 refine syntax a bit, constraints don't need to exist as separate objects
(group-by, order-by to revisit ?)

start using parent
2018-12-27 10:38:21 +00:00
espie
2cc4bc1104 turns out moving "AS" around like this breaks WITH.
give a "parent" handle to list elements because they often need to look
at it.

various syntax additions
2018-12-26 14:01:29 +00:00
espie
b446c7a071 fix stupid double pathid 2018-12-26 13:20:15 +00:00
espie
1c7a76d4f1 choose inserter mode.
pull the AS into the contents so that we can get another mode of creating
views.
2018-12-24 10:49:47 +00:00
espie
f2f8ed867d join will become ambiguous 2018-12-21 17:06:21 +00:00
espie
2259b7166c skip actual autoincrement keys. 2018-12-21 17:06:07 +00:00
espie
b4880eb814 take column names into account 2018-12-21 11:49:05 +00:00
espie
431c00678c register views and tables centrally (so we know the names)
inserter sql statement
2018-12-21 11:11:06 +00:00
espie
3f0b8c5b0e add group_concat idiom
add missing commas
remove extra table name from select if there's just one table involved
2018-12-20 21:02:09 +00:00
espie
e6a5dde19e simplify origin sytnax 2018-12-20 20:39:24 +00:00