Colin Finck

[GETBUILDS] Adapt GetBuilds to the Git future while retaining full support for our existing SVN builds.

- Make use of the new GitInfo library.

- Report the directory in ajax-getfiles-provider.php instead of guessing it from the filename.

- Split up translations into PHP and JS translations.

I didn't update anything other than English given that our website still doesn't do localization.

- Get rid of one of my "WTF ideas", namely PHP-preprocessed JavaScript files.

getbuilds.js.php is now getbuilds.js again, which should increase readability and reduce load times.

- Many code simplifications and consistence with Testman.

    • -0
    • +16
    /trunk/www/www.reactos.org/getbuilds/lang/en.js
    • -0
    • +18
    /trunk/www/www.reactos.org/rosweb/lang/en.js
[ROSWEB] Introduce the GitInfo library, which can be used to query the gitinfo database.

Instead of providing just the known getLatestRevision from RosWeb, this adds getLatestRevisions, getLongHash, getNextRevision, getPreviousRevision, getRevisionInformation, getRevisionRange, and getShortHash.

They allow us to implement convenient features like the Previous/Next button in GetBuilds even though Git doesn't come with linear revision numbers.

It also gives us an easy translation between long and short hashes.

On top of that, ajax-gitinfo.php provides an AJAX interface around the GitInfo library.

    • -0
    • +19
    /trunk/resources/gitinfo/gitinfo.sql
    • -0
    • +114
    /trunk/www/www.reactos.org/rosweb/gitinfo.php
[ROSWEB] Fix a concurrency issue in my AJAX library and adapt it to the new license header style.

[GIT-TOOLS] Add another Git post-receive hook, which maintains a database of the linearized history of master.

On the one hand, this may look like enormous data duplication and an abuse of Git.

On the other hand, this gives our web applications fastest access to Git history, relations between revisions, and commit details.

Some of that isn't even possible with Git itself (like finding out the next revision after a particular one).

We can now adapt GetBuilds and Testman to the Git world and add commit information to any ReactOS web application with ease!

    • -0
    • +88
    /trunk/git-tools/add-gitinfo.php
[GIT-TOOLS] Fix buildbot_sendchange.py to report all revisions between oldrev and newrev and simplify the code.
    • -39
    • +32
    /trunk/git-tools/buildbot_sendchange.py
[GIT-TOOLS] Support multiple repositories and the same GIT_PROJECT_ROOT variable that is used for git-http-backend.

Also fix bugs when testing the script in an Apache/PHP-FPM configuration.

    • -4
    • +5
    /trunk/git-tools/post-receive-webhook.php
[GIT-TOOLS] Add a GitHub Web Hook that updates our Git mirror and calls its "post-receive" hook just like Git would do if the commit was pushed to the mirror repository.

This enables us to set up arbitrary post-receive hooks and keeps our Git setup independent from GitHub.

If we ever moved away from it to a self-hosted Git, all hooks would continue to work without any reconfiguration.

    • -0
    • +62
    /trunk/git-tools/post-receive-webhook.php
[GIT-TOOLS] Adapt "buildbot_sendchange.py" to Git, retaining our ReactOS-specific categorization of rostests.
    • -33
    • +56
    /trunk/git-tools/buildbot_sendchange.py
[GIT-TOOLS] Add a "git-tools" directory, for now with the unmodified SVN version of "buildbot_sendchange.py".
    • -0
    • +47
    /trunk/git-tools/buildbot_sendchange.py
[TESTMAN] Introduce a "comment" parameter to let "submit_result" provide an arbitrary comment instead of always generating it from the build number.
[BUILDBOT] Fully adapt our BuildBot setup to the upcoming Git migration.

- Get a clean version of the source from git://git.reactos.org on every commit and use "git apply" to apply JIRA patches.

- Introduce the naming scheme "reactos-$TYPE-$REVISION_SUFFIX-$UPLOAD_SUFFIX" for all created ISO files.

* $TYPE can be bootcd, livecd, bootcdregtest.

* $REVISION_SUFFIX is the output of "git describe" (e.g. "0.4.7-dev-53-gc2b45a7"), followed by "patch$ID" if a JIRA patch is applied.

* $UPLOAD_SUFFIX is the architecture, compiler, OS, build type identification of the builder.

An example filename that includes everything: "reactos-bootcd-0.4.7-dev-53-gc2b45a7-patch2518-x86-msvc-win-dbg.iso"

- Allow passing arbitrary parameters to CMake in the "configure" script. This is used to pass "-DENABLE_ROSTESTS=1".

- Rename "patching" to "prepare_source", because it also prepares the revision suffix now.

- Introduce "get_suffix" to feed the generated filename suffix back to a BuildBot property (using BuildBot's SetPropertyFromCommand).

- Create one "latest" symlink per upload suffix in "uploadiso".

- Use the "comment" parameter I will introduce in my next Testman commit to supply an arbitrary comment in "submit_result".

Add the JIRA Attachment ID and Reason for patched builds on Timo's request.

- Remove the "revert" step and slim down "clean". A tidy cleanup of the source directory is now being done by the Git build step.

- Remove the "build_host_tools" script and ROS_NO_HOST_TOOLS_REV. You can't build the old revisions with the Git-adapted scripts.

- Work in $ROS_OUTPUT instead of $ROS_OUTPUT/reactos.

- Introduce the UPLOAD_REGTESTCD configuration variable to decide whether to upload the bootcdregtest in "regtestcd".

- Allow us to either download a bootcdregtest or use the latest one from the source folder in "regtest".

- Build rosapps and rostests on the GCCWin and MSVC2010 builder. These don't upload anything, so we can check if the entire tree builds correctly without wasting ISO storage space.

    • -0
    • +27
    /trunk/buildbot/scripts/prepare_source
    • -7
    • +10
    /trunk/buildbot/scripts/submit_result
    • -31
    • +18
    /trunk/buildbot/scripts/uploadiso
    • -0
    • +7
    /trunk/buildbot/scripts/get_suffix
    • -65
    • +44
    /trunk/buildbot/master/master.cfg
  1. … 3 more files in changeset.
[MKISOFS]

Update mkisofs to schily-2017-09-07.

Among other things, this version comes with an upstream fix for David's x64 fix in r75399. We are now using 100% unmodified upstream code again.

[BUILDBOT]

Put one of our best kept secrets under version control, the BuildBot master configuration file.

I've put all sensitive information into an extra "credentials.ini" that is now loaded through a ConfigParser at startup.

Having this in a repository should make it easier to prepare for the Git migration and eventually do the badly required refactoring.

    • -0
    • +290
    /trunk/buildbot/master/master.cfg
    • -0
    • +22
    /trunk/buildbot/master/credentials.ini
    • -0
    • +21
    /trunk/buildbot/master/buildbot.tac
[FS_REC]

Always register CDFS as a low-priority filesystem. This inserts it at the tail of the filesystem queue instead of the head, meaning UDF always takes precedence and UDF/CDFS hybrid media is properly detected as UDF.

Thanks to Pierre for the hint!

Should fix what's reported in https://jira.reactos.org/browse/CORE-9816?focusedCommentId=78820&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-78820

[FS_REC]

- Add a detection routine for CDFS (ISO-9660) volumes that verifies the Primary Volume Descriptor.

- Use this to also detect CDFS on disks and load the CDFS driver if it has not been loaded yet (e.g. when a bootcd/livecd flashed USB drive is inserted at boot of an installed ReactOS).

- Fix a comment in udfs.c.

    • -0
    • +24
    /trunk/reactos/drivers/filesystems/fs_rec/cdfs.h
[USETUP]

Add type 0x96 as "CDFS/ISO-9660" to our list of partition types now that our ISOs flashed to USB use it.

[CDFS]

- Add a basic CDFS filesystem library "cdfslib" for checking and formatting a CDFS volume. Checking is unimplemented and formatting not supported anyway :)

- Use this library in the new "ucdfs" DLL and that DLL in "autochk". Fixes the "Unable to verify a CDFS volume" message at boot.

- Return the right device type (FILE_DEVICE_CD_ROM or FILE_DEVICE_DISK) in when querying volume information in cdfs.sys to get the proper icon in Explorer.

    • -0
    • +5
    /trunk/reactos/dll/win32/ucdfs/ucdfs.rc
    • -0
    • +13
    /trunk/reactos/dll/win32/ucdfs/CMakeLists.txt
    • -0
    • +3
    /trunk/reactos/sdk/lib/fslib/cdfslib/CMakeLists.txt
    • -0
    • +2
    /trunk/reactos/dll/win32/ucdfs/ucdfs.spec
    • -0
    • +1
    /trunk/reactos/dll/win32/CMakeLists.txt
    • -0
    • +1
    /trunk/reactos/sdk/lib/fslib/CMakeLists.txt
    • -0
    • +20
    /trunk/reactos/dll/win32/ucdfs/ucdfs.c
    • -0
    • +37
    /trunk/reactos/sdk/lib/fslib/cdfslib/cdfslib.c
[FORMATTING]

Convert all tabs to spaces in ntdddisk.h

- Add 0x96 as ISO9660 partition type to ntdddisk.h (from https://en.wikipedia.org/wiki/Partition_type) and detect it using IsRecognizedPartition.

- Make our isohybrid create an MBR with partition type 0x96.

This properly assigns a drive letter to the Live-CD and lets me boot into desktop using "qemu -hda livecd.iso".

Fixes CORE-13184

[CDFS]

Fix uninitialized variable usage (why did neither MSVC nor GCC catch this?)

[CDFS]

Register CDFS as a disk filesystem too in order to flash our ISOs directly to USB drives and boot from them. Also fix some DPRINTs in the course of that.

One step closer towards CORE-13184: This gets qemu -hda bootcd.iso to work.

What still doesn't work is qemu -hda livecd.iso and booting from a flashed USB drive on real hardware.

Further changes are also required in fs_rec, but this code isn't ready yet and doesn't seem to be the current blocker either.

Credits go to the entire Hackfest Team for helping on this!!

[FREELDR]

Lower MAX_DISKREADBUFFER_SIZE to let INT 13h, AH=42h only read a maximum of 127 sectors (instead of 128 previously).

This fixes USB booting into the kernel on at least Dell Latitude D531 and Lenovo ThinkPad X61.

Now CORE-13184 remains (in a VM and on real hardware, probably related to the CDFS driver)

[PRINTING]

Apply a consistent style to all similar functions:

- Spoolss functions don't necessarily set ERROR_SUCCESS. Check the boolean return value instead and only query the error code on failure.

- Use AlignRpcPtr and UndoAlignRpcPtr wherever feasible.

- Use one static function per level that can be called from both Get* and Enum* APIs.

- Use PackStrings in all functions that return structures with strings.

- Add TRACEs to all non-static functions in localspl.

- Pass a double pointer to all Marshalling functions and advance to the next structure there.

- Perform sanity checks and return the right error codes in a Windows-compatible way.

- Return TRUE when an Enum* API has no elements to enumerate.

Also fixes CORE-12793

  1. … 4 more files in changeset.
[PRINTING]

Fix all EnumPrintProcessorDatatypes API tests.

[NETAPI32_APITEST]

Fix GCC build

[NETAPI32_APITEST]

Add a simple API Test for netapi32.dll, only covering DsRoleGetPrimaryDomainInformation so far.

It succeeds in ReactOS, but outputs a heap error in the debug log. In real world applications, this later leads to a heap assertion failure.

    • -0
    • +10
    /trunk/rostests/apitests/netapi32/CMakeLists.txt
    • -0
    • +1
    /trunk/rostests/apitests/CMakeLists.txt
    • -0
    • +20
    /trunk/rostests/apitests/netapi32/testlist.c
[PRINTING]

- Create the actual generic handle in _LocalOpenPrinterHandle.

- Fix _RpcClosePrinter call.

- Enable many more tests in winspool_apitest, GetPrinterData.

- Bail out with ERROR_INVALID_PARAMETER for empty strings in _MakePrinterSubKey as well.

- Add the Name registry value for "Dummy Printer on LPT1" to make a test succeed.

This fixes many basic things.. which only got unnoticed, because Printing is only used in the form of API Tests so far.

CORE-13458

CORE-13459

    • -0
    • +1
    /trunk/reactos/boot/bootdata/hivesft.inf
[ADVAPI32_APITEST]

Fix copy-paste mistakes in messages.

That shouldn't have been committed in r75125. Thanks Thomas!
    • -1
    • +1
    /trunk/reactos/boot/bootdata/hivesys.inf
[LOCALSPL]

Add the missing root backslash to the symbolic link path. Fixes CORE-13457