DYN-862: Implement event notification for libraryDataUpdated by sharadkjaiswal · Pull Request #7896 · DynamoDS/Dynamo

sharadkjaiswal · 2017-05-24T06:54:07Z

Purpose

The library view can be updated using various methods such as Installing/Uninstalling packages, creating a custom node, importing zero touch library etc. This PR implements a mechanism to watch the changes in NodeSearchModel, the container of all the library node objects. NodeSearchModel provides notification for every node update. However, when a package is installed or a new library is imported many nodes will be added and this will result in many notifications. This PR implements EventObserver class which can receive event notifications and relay them after applying a transform to the event parameter. It also uses a timer to throttle all the events and raises only one notification for all the changes at a given timespan.

For example an EventObserver observing a stream of events with integer parameter but we want to throttle the notifications and notify with a list of all past values, we can setup the observer as follows:

IEventController controller; // initialised before.
var observer = new EventObserver<int, List<int>>(
        x => controller.RaiseEvent("Numbers", x),
        (x, y) => {
            if (x == null) { x = new List<int>(); }
            x.Add(y);
            return x;
        }
    ).Throttle(TimeSpan.FromMilliseconds(250));
// Setup the event notification
numberGenerator.NumberGenerated += observer.OnEvent;

//Setup dispose callback, unregister OnEvent callback
observer.Disposed += ( )=>{ numberGenerator.NumberGenerated -= observer.OnEvent; };

Declarations

Check these if you believe they are true

The code base is in a better state after this PR
Is documented according to the standards
The level of testing this PR includes is appropriate
User facing strings, if any, are extracted into *.resx files
All tests pass using the self-service CI.
Snapshot of UI changes, if any.
Changes to the API follow Semantic Versioning, and are documented in the API Changes document.

Reviewers

@benglin
@Randy-Ma

User-Zhaoyang · 2017-05-24T09:27:52Z

+                var text = string.Join(", ", items);
+                controller.RaiseEvent("libraryDataUpdated", text);
+                //reset items
+                items.Clear();


There is a chance of risk condition for items. See if at the same time when the timer callback is invoked, new entries are added/removed/updated.

Thanks @Randy-Ma that's a great point, I overlooked it.

User-Zhaoyang · 2017-05-24T09:28:50Z

+            if(controller != null)
+            {
+                model.EntryAdded += OnLibraryDataUpdated;
+                model.EntryRemoved += OnLibraryDataUpdated;


Will all three cases be considered?

yes, all three cases are handled here.

User-Zhaoyang · 2017-05-24T09:30:12Z


+        /// <summary>
+        /// Gets fully qualified name for the given node search element
+        /// </summary>


Minor. But will GetFullyQualifiedName be better for the function name?

benglin · 2017-05-31T05:58:55Z

+        public EventObserver(Action<TResult> observe, Func<TResult, TValue, TResult> reduce)
+        {
+            observer = observe;
+            reducer = reduce;


Haters gonna hate, observers gonna observe, reducers gonna reduce 👍

benglin · 2017-05-31T06:11:33Z

+        /// called directly by passing this object as state.
+        /// </summary>
+        /// <param name="state"></param>
+        protected void Notify(object state)


Was trying to write description for state but it isn't used here.

the state parameter is required for TimerCallback, we don't need to use it.

benglin · 2017-05-31T06:14:12Z

+        public void Dispose()
+        {
+            dispose?.Invoke();
+            dispose = null;


Wow, didn't know this syntax before. Is it equivalent to the following?

if (dispose != null) { dispose?.Invoke(); dispose = null; }

yes, works only with VS2015, BRE self-service builds are failing because of this syntax.

benglin · 2017-05-31T06:35:35Z

+            Assert.AreEqual(3, model.NumElements);
+
+            Assert.IsTrue(resetevent.WaitOne(timeout*3));
+            controller.Verify(c => c.RaiseEvent("libraryDataUpdated", "A, C, E"), Times.Once);


You might have intended to use libraryDataUpdated instead of "libraryDataUpdated" here.

both are same though, but yes I wanted to use the variable rather than a string.

benglin · 2017-05-31T06:46:08Z

+
+            var model = new NodeSearchModel();
+            var controller = new Mock<IEventController>();
+            controller.Setup(c => c.RaiseEvent(It.IsAny<string>(), It.IsAny<object[]>())).Callback(() => resetevent.Set());


Is it right to say that Mock objects executes in a background thread?

Mock objects don't execute in the background thread, but the RaiseEvent method will be called on TimerCallback and that will run in a background thread.

sharadkjaiswal · 2017-05-31T06:54:21Z

+
+        [Test]
+        [Category("UnitTests")]
+        public void RefireTrhottledEvents()


benglin · 2017-05-31T08:56:50Z

+                model.EntryRemoved -= observer.OnEvent;
+                model.EntryUpdated -= observer.OnEvent;
+                observer.Dispose();
+            });


I understand the intent here (and why it is required for the test case below), but it is in my opinion a little harder to understand with AnonymousDisposable. I might have attempted it slightly differently like so:

var observer = new ...; // Subscribe to a new callback even on observer observer.OnDisposing += () => { // Unsubscribe from different events... model.EntryAdded -= observer.OnEvent; // At this point observer internally sets OnDisposing to null... }; return observer;

Would that have caused any issue?

This makes sense, AnonymousDisposable was a fanciful afterthought.

benglin · 2017-05-31T08:57:32Z

+    /// <summary>
+    /// Implements IDisposable to invoke a given dispose action on Dispose().
+    /// </summary>
+    class AnonymousDisposable : IDisposable


DisposableAction? 😛

benglin · 2017-05-31T09:00:13Z

+
+        [Test]
+        [Category("UnitTests")]
+        public void SimpleEvenNumberObserver()


If I'm reading it right, the test case could probably use a name like SimpleEventObserverWithoutThrottle?

Yes, but the throttle is an additional operator, it just observes for even numbers only. Maybe it could be just EvenNumberObserver.

benglin · 2017-05-31T11:08:01Z

I don't have any complaints about this (apart from some difficulties to understand Mock.Verify calls). So looks good!

* Recategorizing built-in methods (#7612) * Recategorize built-in methods * Added methods for List.cs, Math.cs and ImportExport.cs * Updated Math.MapTo * Updated List.cs * Icons and test cases - built-in methods are hidden from user's search - test cases moved to DSCoreNodesImages.resx - minor refinement to some methods - added test cases for List.cs and Math.cs * Set copy local in DynamoCoreWpfTests to false * Updated methods and test cases - Implementation for SetUnion, SetIntersection and SetDifference are the same as built-in methods - Edited ImportExport.ImportFromCSV to reuse the implementation from built-in - Added test cases for ImportExport.ImportFromCSV and fixed other test cases - Consolidated List.Contains with List.ContainsItem - Consolidated NormalizeDepth(list, rank) with NormalizeDepth(list) - Hid some of the built-in methods - Public methods and private helper methods are grouped using regions - Overload of List.Flatten is renamed as FlattenCompletely * Fixes for built in method test cases * Hide SortIndexByValue overload from builtin * Edited List, Math and ImportExport functions - consolidated CSV.ReadFromFile with ImportExport.ImportFromCSV - renamed ImportFromCSV to ImportCSV - moved CSV.WriteToFile to ImportExport.ExportCSV - fixed some of the list functions - reused builtin implememtations for Math.Map and Math.MapTo - moved test cases for CSV functions to ImportExport * migrations for csv.writetofile replacement of CSV.WriteToFile to ImportExport.ExportCSV * migrations for builtin nodes * Renamed FlattenCompletely and removed GetValues * updated icon for list.flatten * Update List.Flatten test case and revert migration for File.ExportToCSV * Consolidate Display nodes for re-categorization * Add one test case for migrating the very old ByGeometryColor node * LibraryViewExtension project for hosted library UI (#7756) * LibraryViewExtension project for hosted library UI * Serve library view with local data * Enable Node creation and console log (#7775) * Cleanup the solution file * Details view implementation over Dynamo canvas (#7780) * Details view implementation over Dynamo canvas * Fix visibility of details view * DetailsView integration (#7788) * Add missing resources. * Tabview to show library view and package list view * Provide Javascript hookup to update active package (#7791) * Fix a typo * Method to load installed packages in JSON format (#7789) * installed packages in json format * updated method to obtain installed pkg * removed getInstalledPackagesJSON() * reverted change to library.html * reverted previous changes * Re-fix a typo * Implement Install Package method on controller (#7793) * Update the bundle for latest updates in the UI. (#7794) * Consolidate some nodes from Logic and Formula to Math (#7757) * move some methods from Logic and Formula to Math * add entries to migration xml * add migration testcase * Remove List.Flatten overload (#7772) * remove list.flatten overload * Updated Dynamo.All.Sln * add migration test case * Update resources and js code from library.js and PMUI (#7796) * Recategorize "ColorRange2D" nodes into "ColorRange" (#7763) * Rename "ColorRange2D" to "ColorRange" * Add migration test case for ColorRange nodes * Install integration and style update (#7797) * Fix the issue that one json string may be invalid (#7802) * Library reorg (#7803) * Added 15 more packages to the list. * updated files to show packages. * Consolidate all file paths nodes under Import/Export category (#7776) * move nodes from file to importexport * add a migration testcase * change category name on ui to Import/Export * revert the slash * change namespace back to DSCore * rename namespace back to DSCore.IO * update loadedtypes.json * Cleanup LibraryReorg branch and remove PMUI code (#7804) * Update the library UI to recategorize two nodes * Consolidate all Directory related nodes to Import/Export Category (#7761) * consolidate directory nodes and rename importexport class * updated DirectoryFromPath and added migrations * change namespace from DSCore.IO to ImportExport * rename namespace and move directory methods to files.cs * add migration test cases * add directory.frompath and update nodemigrationtests * resolve conflicts * updated files * Update layoutSpecs and loadedTypes for ColorRange nodes * add nodes to loadedtype (#7806) * update layoutspecs and loadedtypes * Consolidate Excel nodes under Import/Export category (#7809) * recategorize excel methods * added migration and test case * migration for all excel.write node * move excel and csv methods to dsoffice * remove unnecessary attribute * add migration for DSCore.IO.File.ExportToCSV * Implement custom resource handlers (#7817) * add migration for csv.readfromfile * remove csv.readfromfile icons * Implement resource provider for runtime loadedTypes JSON data (#7826) * Implement resource provider for runtime loadedTypes JSON data * Add search keywords to LoadedTypes * Remove empty keywords * Tooltip implementation for library items (#7822) * Tooltip implementation for library items * Update librarie.min.js to include latest changes * Update LayoutSpecs.json and js to support sections (#7833) * Recategorize two GeometryColor nodes * Add ImportExport.CAD nodes under library (#7832) * Add ImportExport.CAD nodes under library * Restore lost changes * add test case for builtin migration (#7830) * Wrap left-over builtins into DS class for categorization and autocomplete (#7819) * wrap builtins into DS class for categorization and autocomplete * code cleanup * code cleanup * address review comments * addressed review comments * add checks for only static FFI classes to be derived from DS class * code cleanup * code cleanup * added migration, tests * revert changes to loadedtypes json file * updated layout spec json file * add test cases for static class and fix some tests * fixed prototest * Re-recategorize ColorRange nodes (#7840) * updated layoutSpecs.json (#7841) * Refactor the code so the LibraryViewController.cs can be tested (#7834) * Refactor the code so the LibraryViewController.cs can be tested Move methods to seperate class EventController, so this class can be used for mock testing. Derived class object LibraryViewController needs parameters which cannot be mocked. * Add moq test for LibraryViewController * To address the review comments from PR #7834 - Modify AssemblySharedInfo.cs to update CopyRight information of assmblies to 2017 - Remove constructor LibraryViewController() - Revert changes to Dynamo.All.sln realted to visual studio version - Add AssemblySharedInfo.cs to ViewExtensionLibraryTests so the assembly version is set to get the standard versioning * Modify the output path in ViewExtensionLibraryTests.csproj to match the properties from props * Add files related to commit -c03544f * Revert changs to AssemblySharedInfo.cs Modify AssemblySharedInfo.tt to update CopyRight information from 2016 to 2017 * Update librarie.js and other resources (#7842) * Revise category icons for library * add parameters for overloads * removed redundant path * merging new LibUI changes to Dynamo side * Implement IconResourceProvider to get icons for library items (#7869) * Implement IconResourceProvider to get icons for library items * remove methods and fields that are not required * Use category as read from customization xml * address review comments * Add unit tests for ResourceProviders (#7879) * Add unit tests for RespurceProviders * Rename ViewExtensionLibraryTests to LibraryResourceProviderTests - Update json file - Address review comments * Ensure that CopyLocal=False for Dynamo binaries (#7881) * Fix Icon resources for library items (#7880) - Rename BuiltIns.ds to BuiltIn.ds so that the icons for nodes from BuiltIn.ds could be resolved from BuiltIn.customization.dll * Update librarie.js and integrate the new API * Update library minimized javascript file to latest * Fix for DYN-870, removed Keys section from Organize group (#7895) * Remove unused LibraryViewExtension font resources (#7893) * Add description (#7904) * Prevent loading LibraryViewExtension in test mode to prevent NUnit crash (#7906) * wrap builtins into DS class for categorization and autocomplete * code cleanup * code cleanup * address review comments * addressed review comments * add checks for only static FFI classes to be derived from DS class * code cleanup * code cleanup * added migration, tests * revert changes to loadedtypes json file * updated layout spec json file * add test cases for static class and fix some tests * fixed prototest * disable library view extension in test mode * update core UI test to run when test mode is set to true and LibraryViewExtension is loaded * Integrate ImportLibrary UI (#7905) * Make it possible to use an external search function * Update library minimized javascript file to latest * Fix the build error in one test module * Disable right-click on LibraryView (#7913) * DYN-862: Implement event notification for libraryDataUpdated (#7896) * Implement event notification for libraryDataUpdated * update documentation * Event Observer * More testc ases * Don't use old results when throttle re-fires * Address review comments * Addressing review comments * Add node icon for Label.ByPointAndString * Addressing more review comments * fix failing excel tests (#7932) * Fixed failing tests on LibraryReorg branch (#7939) * fix failing excel tests * fixed tests * changed comments * Implement IServiceManager interface (#7949) * build fix, update csproj (#7950) * Add List.Equals overload to List BuiltIn class (#7954) * fix failing excel tests * fixed tests * changed comments * added builtin overload for List.Equals * Remove SearchView and LibrarySearchView WPF views and related tests (#7956) * fix failing excel tests * fixed tests * changed comments * added builtin overload for List.Equals * remove SearchView and LibrarySearchView WPF views * restore deleted comments * Remove v0.0.1 from resource path (#7961) * Fixed merge issues * Create DynamoVisualProgramming.DynamoCoreNodes.nuspec * Include assembly name in fully qualified name (#7971) * Include assembly name in fully qualified name * Create ZeroTouchSearchElement.cs * LibraryViewCustomization Service implementation (#7963) * LibraryViewCustomization Service implementation * Address review comments * Address more review comments * Provides support for default icon when resource not found (#7975) * Provides support for default icon when resource not found * Include concurrency test * Update librarie.js minified and resources (#8004) * Update library resources and categorization (#8010) * Update library resource and categorization * Remove ImportExportTests as its already moved to DSOfficeTests * update librarie.min.js * Implement Resource registration and app shutdown notification mechanism (#8014) * Implement Resource registration and app shutdown notification mechanism * Update LibraryViewExtension.Dispose implementation * Address review comments * Update DSCoreNodes.Migrations.xml removed unnecessary migration path for ExportCSV node

sharadkjaiswal added 2 commits May 24, 2017 14:45

Implement event notification for libraryDataUpdated

baea4e2

update documentation

8177e51

User-Zhaoyang reviewed May 24, 2017

View reviewed changes

sharadkjaiswal added 3 commits May 28, 2017 17:18

Event Observer

04db571

More testc ases

a31b16e

Don't use old results when throttle re-fires

42a51b2

benglin reviewed May 31, 2017

View reviewed changes

sharadkjaiswal commented May 31, 2017

View reviewed changes

benglin reviewed May 31, 2017

View reviewed changes

Address review comments

4b5a3c1

sharadkjaiswal changed the title ~~Implement event notification for libraryDataUpdated~~ DYN-862: Implement event notification for libraryDataUpdated May 31, 2017

sharadkjaiswal merged commit bbb22c7 into DynamoDS:LibraryReorg May 31, 2017

sharadkjaiswal deleted the event branch May 31, 2017 15:29

sharadkjaiswal mentioned this pull request Jun 30, 2017

Implement Resource registration and app shutdown notification mechanism #8014

Merged

7 tasks

Conversation

sharadkjaiswal commented May 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Declarations

Reviewers

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benglin commented May 31, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sharadkjaiswal commented May 24, 2017 •

edited

Loading