Skip to content

Feed aggregator

Fix: Error occurred during a cryptographic operation.

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

Have you ever had this error while switching between projects using the Identity authentication?

Are you still wondering what it is and why it happens?

Clear your cookies. The FedAuth cookie is encrypted using the defined machine key in your web.config. If there is none defined in your web.config, it will use a common one. If the key used to encrypt isn't the same used to decrypt?

Boom goes the dynamite.

Categories: Blogs

Renewed MVP ASP.NET/IIS 2015

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

Well there it goes again. It was just confirmed that I am renewed as an MVP for the next 12 months.

Becoming an MVP is not an easy task. Offline conferences, blogs, Twitter, helping manage a user group. All of this is done in my free time and it requires a lot of time.But I'm so glad to be part of the big MVP family once again!

Thanks to all of you who interacted with me last year, let's do it again this year!

Categories: Blogs

Failed to delete web hosting plan Default: Server farm 'Default' cannot be deleted because it has sites assigned to it

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

So I had this issue where I was moving web apps between hosting plans. As they were all transferred, I wondered why it refused to delete them with this error message.

After a few click left and right and a lot of wasted time, I found this blog post that provides a script to help you debug and the exact explanation as to why it doesn't work.

To make things quick, it's all about "Deployment Slots". Among other things, they have their own serverFarm setting and they will not change when you change their parents in Powershell (haven't tried by the portal).

Here's a copy of the script from Harikharan Krishnaraju for future references:

Switch-AzureMode AzureResourceManager
$Resource = Get-AzureResource

foreach ($item in $Resource)
{
	if ($item.ResourceType -Match "Microsoft.Web/sites/slots")
	{
		$plan=(Get-AzureResource -Name $item.Name -ResourceGroupName $item.ResourceGroupName -ResourceType $item.ResourceType -ParentResource $item.ParentResource -ApiVersion 2014-04-01).Properties.webHostingPlan;
		write-host "WebHostingPlan " $plan " under site " $item.ParentResource " for deployment slot " $item.Name ;
	}

	elseif ($item.ResourceType -Match "Microsoft.Web/sites")
	{
		$plan=(Get-AzureResource -Name $item.Name -ResourceGroupName $item.ResourceGroupName -ResourceType $item.ResourceType -ApiVersion 2014-04-01).Properties.webHostingPlan;
		write-host "WebHostingPlan " $plan " under site " $item.Name ;
	}
}
      
    
Categories: Blogs

Switching Azure Web Apps from one App Service Plan to another

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

So I had to do some change to App Service Plan for one of my client. The first thing I was looking for was to do it under the portal. A few clicks and I'm done!

But before I get into why I need to move one of them, I'll need to tell you about why I needed to move 20 of them.

Consolidating the farm

First, my client had a lot of WebApps deployed left and right in different "Default" ServicePlan. Most were created automatically by scripts or even Visual Studio. Each had different instance size and difference scaling capabilities.

We needed a way to standardize how we scale and especially the size on which we deployed. So we came down with a list of different hosting plans that we needed, the list of apps that would need to be moved and on which hosting plan they currently were.

That list went to 20 web apps to move. The portal wasn't going to cut it. It was time to bring in the big guns.

Powershell

Powershell is the Command Line for Windows. It's powered by awesomeness and cats riding unicorns. It allows you to do thing like remote control Azure, import/export CSV files and so much more.

CSV and Azure is what I needed. Since we built a list of web apps to migrate in Excel, CSV was the way to go.

The Code or rather, The Script

What follows is what is being used. It's heavily inspired of what was found online.

My CSV file has 3 columns: App, ServicePlanSource and ServicePlanDestination. Only two are used for the actual command. I could have made this command more generic but since I was working with apps in EastUS only, well... I didn't need more.

This script should be considered as "Works on my machine". Haven't tested all the edge cases.

Param(
    [Parameter(Mandatory=$True)]
    [string]$filename
)

Switch-AzureMode AzureResourceManager
$rgn = 'Default-Web-EastUS'

$allAppsToMigrate = Import-Csv $filename
foreach($app in $allAppsToMigrate)
{
    if($app.ServicePlanSource -ne $app.ServicePlanDestination)
    {
        $appName = $app.App
		    $source = $app.ServicePlanSource
		    $dest = $app.ServicePlanDestination
        $res = Get-AzureResource -Name $appName -ResourceGroupName $rgn -ResourceType Microsoft.Web/sites -ApiVersion 2014-04-01
        $prop = @{ 'serverFarm' = $dest}
        $res = Set-AzureResource -Name $appName -ResourceGroupName $rgn -ResourceType Microsoft.Web/sites -ApiVersion 2014-04-01 -PropertyObject $prop
        Write-Host "Moved $appName from $source to $dest"
    }
}
    
Categories: Blogs

Microsoft Virtual Academy Links for 2014

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

So I thought that going through a few Microsoft Virtual Academy links could help some of you.

Here are the links I think deserve at least a click. If you find them interesting, let me know!

Categories: Blogs

Temporarily ignore SSL certificate problem in Git under Windows

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

So I've encountered the following issue:

fatal: unable to access 'https://myurl/myproject.git/': SSL certificate problem: unable to get local issuer certificate

Basically, we're working on a local Git Stash project and the certificates changed. While they were working to fix the issues, we had to keep working.

So I know that the server is not compromised (I talked to IT). How do I say "ignore it please"?

Temporary solution

This is because you know they are going to fix it.

PowerShell code:

$env:GIT_SSL_NO_VERIFY = "true"

CMD code:

SET GIT_SSL_NO_VERIFY=true

This will get you up and running as long as you don’t close the command window. This variable will be reset to nothing as soon as you close it.

Permanent solution

Fix your certificates. Oh… you mean it’s self signed and you will forever use that one? Install it on all machines.

Seriously. I won’t show you how to permanently ignore certificates. Fix your certificate situation because trusting ALL certificates without caring if they are valid or not is juts plain dangerous.

Fix it.

NOW.

Categories: Blogs

The Yoda Condition

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

So this will be a short post. I would like to introduce a word in my vocabulary and yours too if it didn't already exist.

First I would like to credit Nathan Smith for teaching me that word this morning. First, the tweet:

Chuckling at "disallowYodaConditions" in JSCS… https://t.co/unhgFdMCrh — Awesome way of describing it. pic.twitter.com/KDPxpdB3UE

— Nathan Smith (@nathansmith) November 12, 2014

So... this made me chuckle.

What is the Yoda Condition?

The Yoda Condition can be summarized into "inverting the parameters compared in a conditional".

Let's say I have this code:

string sky = "blue";if(sky == "blue) {    // do something}

It can be read easily as "If the sky is blue". Now let's put some Yoda into it!

Our code becomes :

string sky = "blue";	if("blue" == sky){    // do something}

Now our code read as "If blue is the sky". And that's why we call it Yoda condition.

Why would I do that?

First, if you're missing an "=" in your code, it will fail at compile time since you can't assign a variable to a literal string. It can also avoid certain null reference error.

What's the cost of doing this then?

Beside getting on the nerves of all the programmers in your team? You reduce the readability of your code by a huge factor.

Each developer on your team will hit a snag on every if since they will have to learn how to speak "Yoda" with your code.

So what should I do?

Avoid it. At all cost. Readability is the most important thing in your code. To be honest, you're not going to be the only guy/girl maintaining that app for years to come. Make it easy for the maintainer and remove that Yoda talk.

The problem this kind of code solve isn't worth the readability you are losing.

Categories: Blogs

Do you have your own Batman Utility Belt?

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38
Just like most of us on any project, you (yes you!) as a developer must have done the same thing over and over again. I'm not talking about coding a controller or accessing the database.

Let's check out some concrete examples shall we?

  • Have you ever setup HTTP Caching properly, created a class for your project and call it done?
  • What about creating a proper Web.config to configure static asset caching?
  • And what about creating a MediaTypeFormatter for handling CSV or some other custom type?
  • What about that BaseController that you rebuild from project to project?
  • And those extension methods that you use ALL the time but rebuild for each projects...

If you answered yes to any of those questions... you are in great risk of having to code those again.

Hell... maybe someone already built them out there. But more often than not, they will be packed with other classes that you are not using. However, most of those projects are open source and will allow you to build your own Batman utility belt!

So once you see that you do something often, start building your utility belt! Grab those open source classes left and right (make sure to follow the licenses!) and start building your own class library.

NuGet

Once you have a good collection that is properly separated in a project and that you seem ready to kick some monkey ass, the only way to go is to use NuGet to pack it together!

Checkout the reference to make sure that you do things properly.

NuGet - Publishing

OK you got a steamy new hot NuGet package that you are ready to use? You can either push it to the main repository if your intention is to share it with the world.

If you are not ready quite yet, there are multiple way to use a NuGet package internally in your company. The easiest? Just create a Share on a server and add it to your package source! As simple as that!

Now just make sure to increment your version number on each release by using the SemVer convention.

Reap the profit

OK, no... not really. You probably won't be money anytime soon with this library. At least not in real money. Where you will gain however is when you are asked to do one of those boring task yet over again in another project or at another client.

The only thing you'll do is import your magic package, use it and boom. This task that they planned would take a whole day? Got finished in minutes.

As you build up your toolkit, more and more task will become easier to accomplish.

The only thing left to consider is what NOT to put in your toolkit.

Last minute warning

If you have an employer, make sure that your contract allows you to reuse code. Some contracts allows you to do that but double check with your employer.

If you are a company, make sure not to bill your client for the time spent building your tool or he might have the right to claim them as his own since you billed him for it.

In case of doubt, double check with a lawyer!

Categories: Blogs

Software Developer Computer Minimum Requirements October 2014

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

I know that Scott Hanselman and Jeff Atwood have already done something similar.

Today, I'm bringing you the minimum specs that are required to do software development on a Windows Machine.

P.S.: If you are building your own desktop, I recommend PCPartPicker.

ProcessorRecommendation

Intel: Intel Core i7-4790K

AMD: AMD FX-9590

Unless you use a lot of software that supports multi-threading, a simple 4 core here will work out for most needs.

MemoryRecommendation

Minimum 8GB. 16GB is better.

My minimum requirement here is 8GB. I run a database engine and Visual Studio. SQL Server can easily take 2Gb with some big queries. If you have extensions installed for Visual Studio, it will quickly raise to 1GB of usage per instance and finally... Chrome. With multiple extensions and multiple pages running... you will quickly reach 4GB.

So get 8GB as the bare minimum. If you are running Virtual Machines, get 16GB. It won't be too much. There's no such thing as too much RAM when doing software development.

Hard-driveRecommendation

512 GB SSD drive

I can't recommend enough an SSD. Most tools that you use on a development machine will require a lot of I/O. Especially random read. When a compiler starts and retrieve all your source code to compile, it will need to read from all those file. Same thing if you have tooling like ReSharper or CodeRush. I/O speed is crucial. This requirement is even more important on a laptop. Traditionally, PC maker put a 5200RPM HDD on a laptop to reduce power usage. However, 5200 RPM while doing development will be felt everywhere.

Get an SSD.

If you need bigger storage (terabytes), you can always get a second hard-drive of the HDD type instead. Slower but capacities are also higher. On most laptop, you will need external storage for this hard drive so make sure it is USB3 compatible.

Graphic Card

Unless you do graphic rendering or are working with graphic tools that require a beast of a card... this is where you will put the less amount of money.

Make sure to get enough of them for your amount of monitors and that they can provide the right resolution/refresh rate.

Monitors

My minimum requirement nowadays is 22 inches. 4K is nice but is not part of the "minimum" requirement. I enjoy a 1920x1080 resolution. If you are buying them for someone else, make sure they can be rotated. Some developers like to have a vertical screen when reading code.

To Laptop or not to Laptop

Some company go Laptop for everyone. Personally, if the development machine never need to be taken out of the building, you can go desktop. You will save a bit on all the required accessories (docking port, wireless mouse, extra charger, etc.).

My personal scenario takes me to clients all over the city as well as doing presentations left and right. Laptop it is for me.

Categories: Blogs

SVG are now supported everywhere, or almost

Decaying Code - Maxime Rouiller - Fri, 04/24/2015 - 20:38

I remember that when I wanted to draw some graphs on a web page, I would normally have 2 solutions

Solution 1 was to have an IMG tag that linked to a server component that would render an image based on some data. Solution 2 was to do Adobe Flash or maybe even some Silverlight.

Problem with Solution 1

The main problem is that it is not interactive. You have an image and there is no way to do drilldown or do anything with it. So unless your content was simple and didn't need any kind of interaction or simply was headed for printing... this solution just wouldn't do.

Problem with Solution 2

While you now get all the interactivity and the beauty of a nice Flash animation and plugin... you lost the benefits of the first solution too. Can't print it if you need it and over that... it required a plugin.

For OSX back in 2009, plugins were the leading cause of browser crash and there is nothing that stops us from believing that similar things aren't true for other browsers.

The second problem is security. A plugin is just another attack vector on your browser and requiring a plugin to display nice graphs seem a bit extreme.

The Solution

The solution is relatively simple. We need a system that allows us to draw lines, curves and what not based on coordinate that we provide it.

That system should of course support colors, font and all the basic HTML features that we know now (including events).

Then came SVG

SVG has been the main specification to drawing anything vector related in a browser since 1999. Even though the specification started at the same time than IE5, it wasn't supported in Internet Explorer until IE9 (12 years later).

The support for SVG is now in all major browsers from Internet Explorer to FireFox and even in your phone.

Chances are that every computer you are using today can render SVG inside your browser.

So what?

SVG as a general rule is under used or thought of something only artists do or that it's too complicated to do.

My recommendation is to start cracking today on using libraries that leverage SVG. By leveraging them, you are setting yourself apart from others and can start offering real business value to your clients right now that others won't be able to.

SVG has been available on all browsers for a while now. It's time we start using it.

Browsers that do not support SVG
  • Internet Explorer 8 and lower
  • Old Android device (2.3 and less), partial support for 3-4.3
References, libraries and others
Categories: Blogs

.NET Innovators

NCover - Code Coverage for .NET Developers - Fri, 04/24/2015 - 18:27

As a community, we are constantly looking for ways to redefine ourselves and improve our products as a whole. Today’s two .NET innovators are helping find those improvements everyday – from building code to teaching and mentoring the future .NET innovators. Let’s celebrate these two .NET rockstars!

Rodrigo Pinto

ncover_mvp_rodrigo_pinto_twitterRodrigo Pinto loves the idea of developing new solutions for and by the .NET community – how can we not celebrate that mindset? With 15 years of experience in software engineering and architecture, Rodrigo is the SharePoint Architect at Everis. He also specializes in PowerShell, Visual Studio ALM, and .NET.

Rodrigo was first recognized as SharePoint MVP in 2011 and has traveled the world training and sharing his expertise at various conferences and speaking engagements. He is always taking on new challenges. Follow along on his .NET adventures on twitter @ScoutmanPT.

Damir Arh

ncover_mvp_damir_arh_twitterDamir has many years of experience with the development and maintenance of complex software solutions for document management. Currently he works as a software architect and is exploring how the latest technologies can be used to improve the development process to better meet user requirements in insurance and banking software.

Damir has been recognized as a .NET MVP since 2012 and also specializes in ASP.NET/IIS, Windows Platform Development, and Visual Studio ALM. He likes to connect with others in the software solutions community through his blog and talks at local conferences and user groups. Follow Damir on twitter @damirarh.

The post .NET Innovators appeared first on NCover.

Categories: Companies

Making It Better: ASP.NET with Visual Basic 14

Visual Web Developer Team Blog - Fri, 04/24/2015 - 14:20

BASIC is part of the DNA of Microsoft, and we have continued to evolve the language for most of the 40 years of the company’s existence.  In the next evolution of the language, Visual Basic 14, there are a number of great features that are being added and we want to make sure that you know we are fully supporting them with ASP.NET in Visual Studio 2015.  Let’s take a look at a few samples of how using Visual Basic with ASP.NET 4.6 makes developers more productive and their code more readable.

 

String Interpolation Makes Web Forms Strings Feel like Razor Syntax

String formatting with Visual Basic is usually a mess.  How many times can you connect strings with ampersand (&) marks and underscores when you span rows?  If you assemble and format your strings by hand, you know what a mess this is.  You already have some nice syntax in razor like the following:

 

<a href="https://twitter.com/@ViewBag.ScreenName/status/@Item.StatusID">

 

If we wanted to do something similar in web forms for this anchor we could end up with a concatenated mess:

 

<a href="https://twitter.com/<%: screenName %>/status/<%#: Item.StatusID %>">

 

Perhaps we got a little more creative and tried to use a String.Format method call:

 

<a href="<%#: String.Format("https://twitter.com/{0}/status/{1}", screenName, Item.StatusID) %>">

 

Yuck… That makes my eyes hurt to read that, with looking back and forth across the string to understand what values are formatted and placed into the output I feel like I’m watching a tennis match.  We know we can do better to make this more readable. With the new string interpolation feature, the string can be decorated with a dollar symbol and then formatted with even simpler template text with a format like the following:

 

<a href="<%#: $"https://twitter.com/{screenName}/status/{Item.StatusID}" %>">

 

Now our string looks more readable, just like the razor syntax without having to scan back and forth across the code.  Values inside of curly braces in our text are interpreted as code values to be inserted at that point in the string, just like our previous String.Format text format but without the annoying back and forth reading for us humans.  This feature isn’t limited to just ASP.NET use, as you can use it in all of the .NET frameworks that support Visual Basic 14.

 

Use the New NameOf Operator in MVC to Validate Arguments

Magic strings are the practice of having string constants in our code that control behavior of our application.  An example of this is when we throw the ArgumentNullException when verifying input parameters:

 

 Public Function TwitterStatus(screenName As String) As ActionResult

If (String.IsNullOrEmpty(screenName)) Then

Throw New ArgumentNullException("screenName")

End If

 

That sample has a code smell, or a deep-seeded problem… if the parameter name changes and an error scenario is triggered, our code will break with this ugliness:

 

As someone who reads stack traces and debugger output, this helpful bit of information that the previous developer attempted to provide for me no longer makes any sense.

The new NameOf operator will take a variable and return a string value for the name of that variable.  Take a look at how our code changes when we use the NameOf operator:

 

 Public Function TwitterStatus(screenName As String) As ActionResult

If (String.IsNullOrEmpty(screenName)) Then

Throw New ArgumentNullException(NameOf(screenName))

End If

 

With that change, we can protect our MVC and WebAPI action methods from inadvertent changes.  Remember, the new refactoring capabilities in Visual Studio (or another refactoring tool that you may be using) will not capture references to the parameter in strings.   If you do use the automatic refactor it will not replace the parameter name in the previous string example, however it will capture this reference to the parameter name. Additionally, our thrown error looks like the following:

 

Now that error statement makes more sense for the next developer who will be maintaining your code.

 

Asynchronous Model Binding in Web Forms

Asynchronous operations have been available with the Task Parallel Library since .NET 4 in 2011.  It’s been around for a long time, and Web Forms just has not had a chance to really feel that love in the more complex interactions that they host. Let’s face it, Web Forms is the event-driven model that could have significant problems if an event is not finished an asynchronous operation when the next event is about to start.  It’s a complex model, but in the latest update to ASP.NET 4.6, we have opened the web forms model to allow Asynchronous Model Binding operations with the Task Parallel library.  Let’s take a look at how easy it is to migrate an existing set of controls to model binds asynchronously.

In this sample, I’m going to load the tweets for both the Visual Studio and ASP.NET twitter accounts into two columns on a web page.  I’ll use the ASP.NET ListView control to output a simple list of the 5 most recent tweets with a syntax and formatting like this:

 

<asp:ListView runat="server" ID="vsTweetList"

SelectMethod="vsTweetList_GetData" ItemType="LinqToTwitter.Status">

<ItemTemplate>

<p>

<%#: $"At: {Item.CreatedAt.ToString("T")} on {Item.CreatedAt.ToString("d")}" %>

<br />

<%#: Item.Text %>

</p>

</ItemTemplate>

<ItemSeparatorTemplate><hr /></ItemSeparatorTemplate>

</asp:ListView>

 

I’m using Model Binding with this control by declaring a SelectMethod on the control.  This method will be called when the control is ready to bind data for presentation, typically after the Page_Load event.  Model Binding for Web Forms is available starting with ASP.NET 4.5, and the syntax is not changed for 4.6 

The ItemType argument on the ListView provides strongly-typed databinding capabilities.  This allows me to avoid the magic-string problem with databinding syntax like Eval(“Status”) If Status was misspelled or did not appear in my data object, I wouldn’t see an error until the page was requested.  By using the ItemType, all of the references to Item in this control are strongly-typed and I get editor intellisense assistance.

Notice I used the new string interpolation feature from Visual Basic in the date format.  While not entirely necessary, it just felt cool to be able to write that as a complete sentence.

The code-behind to fetch the tweets for this presentation would normally look something like this:

 

 Public Function vsTweetList_GetData() As IEnumerable(Of Status)

Return GetTweetsFor("VisualStudio")

End Function

 

That would normally work really well, fetching the data and loading it into my resultant HTML very quickly.  In this case, I want to load two sets of tweets.  That means two requests to Twitter to load data and blocking page processing while those two requests are fulfilled by the Twitter APIs.

We can make these requests asynchronous by changing the syntax of the specified SelectMethod to return a Task.  We also need to mark the @Page directive with an async=”true” attribute.  It seems too simple, but the web forms framework was updated to allow this type of simple update to “just let the magic happen”

 

 Public Async Function vsTweetList_GetDataAsync() _

As Threading.Tasks.Task(Of IEnumerable(Of Status))

Return Await GetTweetsForAsync("VisualStudio")

End Function

 

Seriously, that’s all that you need to do in order to allow your requests to fire asynchronously.  Just add the async keyword, return a task, and await the asynchronous method call.  Now, both of my requests to the Twitter API will fire at the same time, the ASP.NET runtime will manage the threads that call the API and ensure that they are all processed before returning the page to the requesting user.

 

Roslyn Support for Visual Basic Compilation

Graph of comparison between compiler versionsWith the introduction of the new Roslyn compiler,  you now have a turbo-charged ASP.NET compiler experience.  In the tests in our lab, we have measured an almost 50% speed increase for Visual Basic developers at compile time.  However, we didn’t stop there.

Since May 2014, there has been a NuGet package available called “CodeDOM Providers for .NET Compiler Platform (Roslyn)” and wow was that hard to find.  Only 600 downloads of this package have been recorded at the time of this article’s writing, and that’s a real shame.  This package activates the Roslyn compiler for ASPX page parsing and compiling.  Do you remember waiting for the compiler to re-interpret your ASPX pages each time you change HTML formatting?  This package cuts that time significantly, but is in pre-release mode and only works for C#-based ASPX files.  With the coming release of Visual Studio 2015, we are going to release a 1.0 version of this package and add Visual Basic support.

 

ASP.NET 5 – C# Support And Also Visual Basic

We’ve talked about ASP.NET 5 as a major update of the ASP.NET framework with Roslyn and cross-platform support in mind since our initial public discussions.  It is not a short path, and we focused initially on completing support for C#.  In the months since our initial announcements, we have heard from many of you, telling us how much you like Visual Basic and that they want to see support for it in ASP.NET 5. 

We are excited today to announce that ASP.NET 5 will have full support with Visual Basic (both tooling and runtime – including cross platform runtime support).  As always, we will continue this development of ASP.NET 5 in the open, and you can track our progress or even contribute on GitHub at http://github.com/aspnet/home.

 

Summary

Visual Basic, ASP.NET and even Classic ASP before that have had a long history together.  We’re committed to that partnership, and we will continue to evolve the Visual Basic story with ASP.NET.

You can learn more about the new features in Visual Basic 14 from Lucian Wischik and from Kathleen Dollard in a pair of snack-sized five-minute videos.

Categories: Companies

Work stealing in the presence of startup / shutdown costs

Ayende @ Rahien - Fri, 04/24/2015 - 10:00

I mentioned that we have created our own thread pool implementation in RavenDB to handle our specific needs. A common scenario that ended up quite costly for us was the notion of parallelizing similar work.

For example, I have 15,000 documents to index .That means that we need to go over each of the documents and apply the indexing function. That is an embarrassingly parallel task. So that is quite easy. One easy way to do that would be to do something like this:

foreach(var doc in docsToIndex)
	ThreadPool.QueueUserWorkItem(()=> IndexFunc(new[]{doc}));

Of course, that generates 15,000 entries for the thread pool, but that is fine.

Except that there is an issue here, we need to do stuff to the result of the indexing. Namely, write them to the index. That means that even though we can parallelize the work, we still have non trivial amount of startup & shutdown costs. Just running the code like this would actually be much slower than running it in single threaded mode.

So, let us try a slightly better method:

foreach(var partition in docsToIndex.Partition(docsToIndex.Length / Environment.ProcessorCount))
	ThreadPool.QueueUserWorkItem(()=> IndexFunc(partition));

If my machine has 8 cores, then this will queue 8 tasks to the thread pool, each indexing just under 2,000 documents. Which is pretty much what we have been doing until now.

Except that this means that we have to incur the startup/shutdown costs a minimum of 8 times.

A better way is here:

ConcurrentQueue<ArraySegment<JsonDocument>> partitions = docsToIndex.Partition(docsToIndex.Length / Environment.ProcessorCount);
for(var i = 0; i < Environment.ProcessorCount; i++) 
{
	ThreadPool.QueueUserWorkItem(()=> {
		ArraySegment<JsonDocument> first;
		if(partitions.TryTake(out first) == false)
			return;

		IndexFunc(Pull(first, partitions));
	});
}

IEnumerable<JsonDocument> Pull(ArraySegment<JsonDocument> first, ConcurrentQueue<ArraySegment<JsonDocument>> partitions )
{
	while(true)
	{
		for(var i = 0; i < first.Count; i++)
			yield return first.Array[i+first.Start];

		if(partitions.TryTake(out first) == false)
			break;
	}
}

Now something interesting is going to happen, we are scheduling 8 tasks, as before, but instead of allocating 8 static partitions, we are saying that when you start running, you’ll get a partition of the data, which you’ll go ahead and process. When you are done with that, you’ll try to get a new partition, in the same context. So you don’t have to worry about new startup/shutdown costs.

Even more interesting, it is quite possible (and common) for those tasks to be done with by the time we end up executing some of them. (All the index is already done but we still have a task for it that didn’t get a chance to run.) In that case we exit early, and incur no costs.

The fun thing about this method is what happens under the load when you have multiple indexes running. In that case, we’ll be running this for each of the indexes. It is quite likely that each core will be running a single index. Some indexes are going to be faster than the others, and complete first, consuming all the documents that they were told to do. That means that the tasks belonging to those indexes will exit early, freeing those cores to run the code relevant for the slower indexes, which hasn’t completed yet.

This gives us dynamic resource allocation. The more costly indexes get to run on more cores, while we don’t have to pay the startup / shutdown costs for the fast indexes.

Categories: Blogs

Convert Part of Excel to Image Using Free Excel API

C-Sharpcorner - Latest Articles - Fri, 04/24/2015 - 08:00
This article shows how to convert part of an Excel worksheet to an image using the free Spire.XLS API.
Categories: Communities

Gift For Developers From SharePoint

C-Sharpcorner - Latest Articles - Fri, 04/24/2015 - 08:00
In this article we will explore the options provided by Content Editor Web part and how it is helpful for developers.
Categories: Communities

Xamarin Guide 8: Change the App.cs to App.xaml

C-Sharpcorner - Latest Articles - Fri, 04/24/2015 - 08:00
In the Xamarin workshop series, today we learn how to change the App.cs to App.xaml.
Categories: Communities

ServiceController Class in Windows Service

C-Sharpcorner - Latest Articles - Fri, 04/24/2015 - 08:00
This articles explains the ServiceController.
Categories: Communities

SharePoint: PowerShell to Get&#47;Add&#47;Remove Users of Local Administrator Group

C-Sharpcorner - Latest Articles - Fri, 04/24/2015 - 08:00
This article outlines how to get, add and remove users of a local administrator group on SharePoint servers using a PowerShell script.
Categories: Communities

Returning JavaScript: Undefined Marries NaN

C-Sharpcorner - Latest Articles - Fri, 04/24/2015 - 08:00
In this article we will learn about returning JavaScript Undefined and NaN.
Categories: Communities

Null Coalescing (??) Operator in C#

C-Sharpcorner - Latest Articles - Fri, 04/24/2015 - 08:00
In this article we will learn about Null Coalescing (??) Operator in C# programming.
Categories: Communities