getTP -- Get Source Code TODO's for a TaskPaper formatted file

25 Jun 2017

This is a quick and simple Python script that uses Ag – The Silver Searcher to find ‘TODO’ items in a source tree.

The output is formatted for the TaskPaper plain text format and sent to stdout. The output can be piped to a .taskpaper file.

Nothing fancy. Ag is super fast and does the heavy lifting. The Python code formats the Ag format to TaskPaper.

The current directory name is used for the root project name.
Each file with TODO’s are sub-projects. The sub-project names include relative directories.
Tasks begin with the line number of the TODO.

  gettp:

    getTP.py:
      - line: 2    Combine TODO's to a TaskPaper list
      - line: 8    todo_command = 'ag TODO'
      - line: 13   Use Ag Silver Searcher to find TODO comments in source

    README.md:
      - line: 1    # getTP - Get Source Code TODO's for a TaskPaper formatted file
      - line: 3    This is a quick and simple Python script that uses [Ag - The Silver Searcher]...

    sample//test.md:
      - line: 5    * TODO: Ad nulla ad Lorem quis et quis in in laborum incididunt adipisicing occaecat id voluptate.

BFG Repo-Cleaner

30 May 2017

Removes large or troublesome blobs like git-filter-branch does, but faster. And written in Scala.


bfg --strip-blobs-bigger-than 100M --replace-text banned.txt repo.git

Replace all passwords and listed in a file (prefix lines ‘regex:’; or ‘glob:’; if required) with ***REMOVED*** wherever they occur in your repository :


bfg --replace-text passwords.txt  my-repo.git

Delete all files named ‘id_rsa’ or id_dsa’ :

bfg --delete-files id_{dsa,rsa} my-repo.git

Using BFG on Windows

Use Git Bash or Windows 10 Bash.
Download and install Java from here.
Download BFG here.
Make an alias for BFG:


java -jar bfg.jar $*

Make a replace.txt file:

PASSWORD        # Replaces PASSWORD with ***REMOVED***
PASSWORD==>TEST # Replaces PASSWORD with TEST
PASSWORD==>     # Replaces PASSWORD with blank
                # You can use Regular Expressions also

Run this command in the repo folder from Git Bash or Win10 Bash:


bfg --replace-text replace.txt

When bfg is done run these commands

git reflog expire -expire=now -all && git gc -prune=now -- aggressive

git push                   # -f may be needed

Notes

I had already pushed the repo and shared it. That got complicated by multiple branches. My fault. It would have been much simpler to run bfg before sharing.
I fumbled a bit trying to get bfg running in Windows command shell.

Resources

Microsoft Windows File Server Locked File Watcher

29 May 2017

It is common for companies to work with Excel, Word and other files on Windows network shared folders.
More than one user cannot usually work on files like Excel at the same time.
When trying to open a locked network file it can be hard to see who has it locked.
The best method to see who has a file locked is from the Windows file server. But that requires Administrator level access. That level of access is not practical for all users for security reasons.
This tool has two parts:
A Powershell process that watches for locked files and saves the result.
A web server that displays the locked files and who has them locked.
No special access is required to view the web results.The Powershell process uses
The Powershell process uses a list of watched files and folders. This limits the number of files displayed. This also allows it to avoid showing sensitive files or network folders.

Visit Github to download.

Cliplink -- a utility for short, reliable links on Windows networks

06 Apr 2017

What is Cliplink for

Users wanted a way to create reliable links to files and folders on Windows network drives.
Spaces and special characters break Outlook’s hyperlinks to network files. Clipboard-Link fixes that problem.

How do I get set up

Get Cliplink from => GitHub.
There is no configuration
There are no dependencies
No database configuration
Save the contents of this repo to a network drive or share.
Tell users to run setup from there.

How does it work

Locate the file or folder on the network.
Right-click and select Send To > clipboard-link
Cliplink creates a safe link to the file and puts it on the Clipboard.
Paste the link into Email, documents, etc.

To make a friendly link

Create a link to a network file or folder with Clipboard-link and paste into a message
Select the link part and Cut
Select the name part between the brackets
Select Insert menu > Hyperlink
Paste the link in the URL box
Click Ok
Clipboard-link copies the file/folder name and shortened link to the clipboard

  [Engineering Documentation]                  file://F:\ENGINE~1

          ^ Name ^                               ^   link  ^

NOTE: **Cliplink will not create links for executables [files ending with EXE, COM, BAT, etc.].

Confluence, a Great Documentation Tool

06 Apr 2017

Introduction

Years ago, I was the new system administrator at the company. I had to get a handle on the multitude of systems, servers, and software. My predecessors did a great job of documenting the details in Microsoft Word. Because of the number of documents, it was hard to find stuff. It was time-consuming to keep details up to date as software and systems changed.

Then I discovered MediaWiki, the Open Source software that runs Wikipedia. An old donated PC was the host for Linux and MediaWiki. I copied the Word documents into the wiki. It was easy and natural (for me at least) to build easy to navigate links to the IT documentation. MediaWiki keeps older versions so it is easy to see the history of changes. Wiki markup is weird and unnatural to normal people but a few folks at the company took to it. The help was welcome for maintaining the IT processes and procedures. Contributors logged in to MediaWiki so we could see who made changes.

Unlike the Word documents, it was easy to search and find information. We could email hyperlinks to folks in need of support. That saved time for me and other power users. And normal folks would sometimes let us know when documentation was out of date or wrong. MediaWiki reduced the friction of sharing and managing the documentation.

I looked at SharePoint a few times. SharePoint is well integrated with Windows and Microsoft Office products. It seems like a natural extension for folks familiar with that world. SharePoint’s wiki didn’t work for me though. And the easy extensibility worked against using it for documentation. That lead to a lot of rabbit holes to get lost in when the focus should be elsewhere. Don’t get me wrong, I am very good in the Microsoft world but it felt like more friction to get things done.

Atlassian Confluence

I came across Atlassian Confluence while exploring Agile development methods. Confluence is like a polished MediaWiki.

Working together on documentation is well thought out. Instead of emailing the author about changes a user selects text on the Confluence page. A popup presents the user with the option to comment right on the page. Comments are part of the documentation. That is much better than email silos.

The real-time collaboration is great. When more than one user edits a page at the same time the changes synchronize between them in real time.

Like MediaWiki, Confluence keeps track of previous versions of the pages. Having that history gives users more confidence. They do not have to maintain many copies of a document. The history is all right there with that page in Confluence.

Embedding and attaching pictures is easy in Confluence. Users can also attach Word, Excel, PDFs and other files.

A cool plugin for Confluence is Gliffy. It is like a web based Visio for creating diagrams, flowcharts and so forth. Many times IT documentation is better with a network diagram or a business workflow. The Gliffy plugin tracks versions as well. Users have that history of changes which helps with complex diagrams.

Confluence benefits from tight integration with Atlassian JIRA. That is good if you use JIRA. JIRA is an issue tracking tool that is great for software development. Many use JIRA for business project management and managing service.

Confluence and JIRA are both available as cloud services from Atlassian. No servers to maintain.

Having time on my hands I went with installing Confluence and JIRA on my servers. Well, actually Hyper-V virtual machines running Linux. The install process is well documented. I ran into a few problems but Atlassian had intuitive help to get through them. The price made me smile. My self-hosted servers were $10 each for 10 users. That’s a good value and a low barrier for these tools.

Confluence helps my efficiency with the DRY principle (Don’t Repeat Yourself). Having the notes and documentation online means they’re available almost everywhere. And Confluence reduces the friction of maintaining documentation.