Wednesday, April 27, 2005

 

Some notes on DotNet decompiler - 3

1.

现在的玩法,不自己搞个虚拟机都不好意思

http://www.oreans.com/codevirtualizer.php

Tuesday, April 26, 2005

 

Some notes on DotNet decompiler - 2

I.
MaxtoCode原理探讨

这边还是放点随手翻到的东西吧

I.
from http://www.cnblogs.com/rick/archive/2006/07/17/453150.html

这里研究的对象是 MaxtoCode 3.1试用版.这里只探讨程序代码的加密.

对.Net程序代码的加密过程如下:

1. 运行 ildasm 将程序集反编译成 il代码文件.

2. 对IL代码文件进行处理.(*)

3. 运行 ilasm 将 IL代码文件编译成程序文件.

4. 直接对程序文件中的il字节码加密.(**)

粗体表示的 2 , 4 是关键步骤.

我们先来看看第四步.这一步就是加密的关键步骤,这里就是使用MaxtoCode的加密算法对程序代码进行加密。

显然,对于破解来说最直接直观的方法就是对其第四步的逆向解密。

如果从这个方向去破解解密加密过的程序,那就像MaxtoCode号称的那样MAXTOCODE的强度建立在加密算法之上。

理论上方法是可行的,但是工作量是非常大的。

那么我们还有其它的路可行呢?

现在来看看第二步MaxtoCode都做了什么。

用vs2003建一个最简单的winform程序,然后用MaxtoCode加密试试。我们将第三步之后,第四步之前的exe文件拿来研究。这个时候的exe程序代码是还没有被加密的。可以reflector。

看看 这个exe和我们直接的exe有什么区别:

1. 增加了一个类InFaceMaxtoCode .

2. 类都被增加了一个静态构造函数,在这个函数里面调用了InFaceMaxtoCode的一个静态函数Startup。

3. 类的原有构造函数里面也增加了调用InFaceMaxtoCode.Startup的语句。

从这些来看,MaxtoCode的目的是要确保InFaceMaxtoCode.Startup 在程序中能够最早的运行。

这个行为和win32程序加壳很像,一般壳都是加密程序代码,然后修改程序的启动入口,首先执行壳的代码,完成程序的解密,然后再执行程序。一般壳有一个特点:加密是对整个程序,启动时也是整个程序完全解密,然后再执行。(我也见到过一个很特别的壳,程序是部分解密的,软件注册算法的那一块, 是执行一部分解密一部分,然后之前解密的又被垃圾信息填充了。)

对于壳只要我们找对了时间和地点,就能从内存中得到我们需要的东西。

那么 MaxtoCode加密后的。Net程序呢?

先来看看 MaxtoCode的加密方式。用ildasm反编译 加密后的程序,会报很多错误,这是正常的,从生产的IL文件看,各个类,函数都还在,只是函数体里面是只有ildasm的错误信息。显然是加密后的代码无法反编译。MaxtoCode对。Net程序的加密不是对程序整体的,而只是对函数体加密,程序类结构不变。有一点我们是很清楚的,加密后的程序要能够正常运行,在运行时肯定是需要解密的。而解密的关键就在InFaceMaxtoCode.Startup 里面。

现在我们来看看InFaceMaxtoCode.Startup 里面究竟做了什么。InFaceMaxtoCode 类的代码如下:

using System;
using System.IO;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Text;

public class InFaceMaxtoCode
{
static InFaceMaxtoCode()
{
InFaceMaxtoCode.started = false;
}

[DllImport("MRuntime3.dll", EntryPoint="CheckRuntime", CharSet=CharSet.Unicode, SetLastError=true, ExactSpelling=true)]
private static extern int A______();
[DllImport("KERNEL32.DLL", EntryPoint="GetModuleHandleA", CharSet=CharSet.Ansi, SetLastError=true, ExactSpelling=true)]
private static extern int B______(string x13d52f7d8e232e61);
private static string ByteToString(byte[] x5fc6100148519126)
{
return Encoding.ASCII.GetString(x5fc6100148519126);
}

[DllImport("MRuntime3.dll", EntryPoint="MainDLL", CharSet=CharSet.Ansi, SetLastError=true, ExactSpelling=true)]
private static extern bool C______(int x19218ffab70283ef, int xe7ebe10fa44d8d49);
[DllImport("KERNEL32.DLL", EntryPoint="SetEnvironmentVariableA", CharSet=CharSet.Ansi, SetLastError=true, ExactSpelling=true)]
private static extern bool D______(string x427bb0e14ed9e9b1, string x84ee6c5b88919f4c);
public static void Startup()
{
if (!InFaceMaxtoCode.started)
{
string text1 = "";
string text2 = "MRuntime3.dll";
if (AppDomain.CurrentDomain.RelativeSearchPath != null)
{
if (AppDomain.CurrentDomain.RelativeSearchPath.IndexOf(@":\") != -1)
{
text1 = AppDomain.CurrentDomain.RelativeSearchPath;
}
else
{
text1 = AppDomain.CurrentDomain.BaseDirectory + AppDomain.CurrentDomain.RelativeSearchPath;
}
}
else
{
text1 = AppDomain.CurrentDomain.BaseDirectory;
}
string text3 = Environment.GetEnvironmentVariable("path");
if (text3.IndexOf(text1) == -1)
{
InFaceMaxtoCode.D______("path", text3 + ";" + text1.Replace("/", @"\"));
}
if (text1.Substring(text1.Length - 1, 1) == @"\")
{
text1 = text1;
}
else
{
text1 = text1 + @"\";
}
if (File.Exists(text1 + text2) && !File.Exists(Path.GetTempPath() + text2))
{
File.Copy(text1 + text2, Path.GetTempPath() + text2);
}
if (text3.IndexOf(Path.GetTempPath()) == -1)
{
InFaceMaxtoCode.D______("path", text3 + ";" + Path.GetTempPath().Replace("/", @"\"));
}
int num1 = 5;
num1 = InFaceMaxtoCode.A______();
if (num1 == 0)
{
int num2 = InFaceMaxtoCode.B______(text2);
int num3 = InFaceMaxtoCode.B______(Assembly.GetExecutingAssembly().Location);
InFaceMaxtoCode.started = InFaceMaxtoCode.C______(num2, num3);
}
else
{
//一堆垃圾代码,报告启动错误信息的。
}
}
}

private static bool started;
}


Startup精简后的代码如下:
public static void Startup()
{
if (!InFaceMaxtoCode.started)
{
//准备运行库;
int num1 = 5;
num1 = InFaceMaxtoCode.A______();
if (num1 == 0)
{
int num2 = InFaceMaxtoCode.B______(text2);
int num3 = InFaceMaxtoCode.B______(Assembly.GetExecutingAssembly().Location);
InFaceMaxtoCode.started = InFaceMaxtoCode.C______(num2, num3);
}
else
{
//一堆垃圾代码,报告启动错误信息的。
}
}

从代码里面我们看得到InFaceMaxtoCode.Startup 正常启动后,在整个程序集中只会运行一次。

关键函数是 运行库的MainDLL,这个函数有两个参数,一个是运行库的句柄,一个是程序集的句柄。这个句柄实际上就是程序在内存中加载的位置。MaxtoCode加密后的程序都是对齐到0x1000的。

II.
http://www.cnblogs.com/rick/archive/2006/09/14/504525.html

自上次写第一篇文章到现在不知不觉两个月过去了,这篇文章我们将介绍怎么获取解密后的IL字节代码。
我们先回顾一下前文,在上一回我们提到“InFaceMaxtoCode.Startup 正常启动后,在整个程序集中只会运行一次。”。
当时这种说法是很武断的,如果 “InFaceMaxtoCode.C______(num2, num3)” 的返回值总是 false的话,该函数就会被执行多次,
不过根据后来动态调试的结果,我们证实了“InFaceMaxtoCode.C______(num2, num3)” 的返回值为 true,因此上次的说法是正确的。

现在言归正传,怎么取得解密后的代码呢?大概两个方向,
1.正面交锋,直接攻破maxtocode的运行库。
这就将问题直接回到了传统的win32层面,不过这个东西是业内人士写的在这方面的保护工作做得很好,像我这样的菜鸟就很难直接攻破了。
我曾有一个设想,就是通过分析运行库找到解密函数的入口,然后弄一个stub dll,hook这个地方,把解密后的il代码dump出来。
实际跟踪几次后我就放弃了。从跟踪到的信息来看,我猜测,运行库是通过 mscorwks.dll 挂接到 jit,在jit的前面实时解密代码。
理论上我们也可以挂一个到jit前面,在那里dump解密的il代码,不过这个实现的方式,还不清楚,如果弄明白了,也就能写一个同样原理的加密软件了。
这个难度比较大,所以我最终放弃了这个方案。

2.避开运行库,我们直接利用dotNet 2.0的特性获取IL代码。
如是我就试着用2.0写了一个winform程序,加密,运行,发现报错。
maxtocode3.1不支持2.0的winform程序,这就使我的这个方案实验夭折了。
两个月过去了,发现maxtocode升级到3.11了修正了这个bug,今天终于可以继续实验了。

我们来建一个简单的winform程序。一个窗体,然后一个按钮。
代码如下:

1 using System;
2 using System.Collections.Generic;
3 using System.ComponentModel;
4 using System.Data;
5 using System.Drawing;
6 using System.Text;
7 using System.Windows.Forms;
8 using System.Reflection;
9 using Spaces;
10 namespace Test5
11 {
12 public partial class Form1 : Form
13 {
14 public Form1()
15 {
16 InitializeComponent();
17 }
18
19 private void TestMethod()
20 {
21 // [7/17/2006]
22 int i = 0;
23 i = 1;
24 i++;
25 if(i>0)
26 {
27 MessageBox.Show("OK");
28 }
29 }
30
31 private void button1_Click(object sender, EventArgs e)
32 {
33 Type tp = this.GetType();
34
35 MethodInfo mi = tp.GetMethod("TestMethod",
36 BindingFlags.NonPublic|BindingFlags.DeclaredOnly|
37 BindingFlags.Public|BindingFlags.Static
38 |BindingFlags.Instance);
39 if(mi == null)
40 {
41 MessageBox.Show("err");
42 return;
43 }
44 MethodBody mb = mi.GetMethodBody();
45 byte[] bt= mb.GetILAsByteArray();
46 StringBuilder sb = new StringBuilder();
47 for (int i = 0; i < bt.Length; i++)
48 {
49 sb.Append(bt[i].ToString("X2"));
50 sb.Append(" ");
51 }
52 string stxt = sb.ToString();
53 MessageBox.Show(stxt);
54
55 }
56
57 }
58 }

编译运行,我们点击按钮就能看到 TestMethod 的IL字节码。
然后用maxtocode加密在运行,同样能看到 TestMethod 的IL字节码。
两次看到的结果一样的,这个是当然了,如果不一样,maxtocode就破坏了程序的正确性了。

好了,我们的实验成功了。

看到这里大家应该知道怎么获取解密后的IL代码了吧。

这种方式比在内存里面找代码或者hook到maxtocode解密后dump代码的方式要优越很多,
因为内存dump的方式你还要担心运行时的函数覆盖率,没有运行到的就dump不到。

这种方式我们利用 DotNet的反射机制,可以枚举出程序集中的所有类型,以及每个类型的所有方法,成员,字段,构造函数等。

初步实验了一下,对于加了密的dll文件还是比较好弄的,2.0的、1.1的都能弄出IL代码来。
对于exe文件还有一关需要解决,那就是如何将我们的DotNet dll程序集插入到exe的运行空间中去。

今回就先到这里了,下回再实际写程序演练获取解密后的IL字节代码。

III.
From
http://www.cnblogs.com/rick/archive/2006/09/28/517603.html
http://www.cnblogs.com/rick/archive/2006/09/29/518756.html
http://www.cnblogs.com/rick/archive/2006/10/14/528690.html

上一回我们试验了通过反射的方式获取method的源代码。这次我们就用一个实例来演示dump一个程序集中的所有类型和方法的IL源代码。编译这个程序,运行,dump出il字节码,然后拿 maxtocode加密。再运行,dump出il字节码,然后找一个method 如 button1_click,比较一下他们的IL字节码是否一样。当然结果应该是一样的。

这里主要有三个关键函数
private void DumpAssembly(Assembly ass, string path);
private void DumpMethod(MethodBase mb, StreamWriter sw);
private void DumpType(Type tp, StreamWriter sw);
这三个就是一个例子演示如何dump整个程序集。

如要dump 一个加密的dll,我们就可以直接用这个程序来改,
首先添加引用,引用那个dll,然后随便实例话一个该dll中的type。
然后获取该dll的 Assembly 对象,再调用DumpAssembly函数即可。

在前面几章我们已经能够去掉被加密程序原始的IL字节码了。这些字节码是十六进制的,我人脑直接来阅读是非常困难的。这一章主要介绍将字节码翻译成 可阅读的 MSIL 汇编代码,以及前几章的遗留问题解决。


这里我们将用到上面这个工具软件 IlByteDecoder.

软件下载地址:http://www.bbsftp.com/temp/ILByteDecode.rar
使用比较简单,注意中间那个 文件名 一项,这个可以填也可以不填,如果没有填的话,
解码出来的 msil 代码中将无法显示字符串值和方法名称。

在前面提到的 对Exe程序的注入问题,现在已经找到的解决方案:
1。传统win32注入方式,采用C++/CLI 编写dll 注入。
2。profile 方式 modify IL on the fly ,直接注入dot net dll。

注入之后,就可以直接在内存里面操作,

tankaiha: "实践证明被MaxtoCode加密的软件(这里用的CodeLib)原代码可以在内存中还原。"

元数据还原以及IL解码的改进

前一回讲了 IL字节码的解码问题,并提供了一个小工具,但解码的效果和 ildasm还是差很多,给阅读也带来了一些困难。还有就是有些文件选择文件后解码会出错,这是因为maxtocode对文件里面的元数据进行了随机加密。这一回主要解决元数据的还原以及对解码进行改进。

题外话:国庆后maxtocode推出了3.12版,称对.net formwork 2.0获取msil代码的方式进行了限制。即在不作任何改进的情况下,我们前面介绍的方法将无法取得IL字节码。dreaman已经找到了取消这个限制的方法了,不久tankaiha 就会整合完成新的 injectReflector 。就是前一回贴的第二张图片,该工具已经在看雪发布了。

言归正卷,maxtocode对net程序加密时还能对元数据进行随机部分加密,即破坏静态元数据的完整性。直接磁盘文件里面读取的元数据是不完全正确的,所以导致了解码程序取元数据信息时出错。解码程序现在已经更新处理了这样的异常。同时增加了元数据还原的功能。怎么还原?这个比il字节码要容易,根据元数据的特性,程序运行后在内存中必然有完整的元数据,我们所要做的就是从内存中直接dump出来就ok了。

元数据在内存中的位置可以从PE的CLI Header中取得,CLI Header的位置有可以从PE可选头部中取得。首先参考PE文件结构取得PE文件的可选头部。在这个结构中有一个成员是 DataDirectory 数组。其中DataDirectory[14] 就是记录的 CLI Header的偏移和大小。在CLI Header结构:

struct _CLIHeader
{
DWORD cbSize;//size of 72
INT16 nMajor;//2
INT16 nMinor;//0
DWORD mdRVA; //元数据在内存中的便宜量
DWORD mdSize;//元数据的大小
DWORD dwFlags;
DWORD tkEntry; //mdtMethodDef
INT64 Resources;
INT64 SrongNameSig;
INT64 CodeManagerTable;//0
INT64 VTableFixedup;
INT64 ExprotTableJump;//0
INT64 ManagedNativeHeader;//0}
在这个结构里面就能找到元数据的偏移量和大小。

dump的功能已经增加到新版的ilbytedecoder中了。这个界面上上一回的界面,解码的是同一段IL字节码。效果已经类似ildasm的了,基本上可阅读了。同时提供了选择 Raw MetaData的进行解码的功能,这样就不用担心因为原文件的元数据被破坏导致解码不正常了。程序新加的dump功能可以将内存中的元数据保存为 Raw MetaData文件中。

程序新加的dump功能可以将内存中的元数据保存为 Raw MetaData文件中。下载地址:http://www.bbsftp.com/temp/ILByteDecoderV1.5.rar

Monday, April 25, 2005

 

Some notes on DotNet decompiler - 1

1.
谈谈ILDasm的功能限制与解除

By aiasted
From http://www.cnblogs.com/aiasted/archive/2005/05/05/149639.html

首先,我在此申明,此文并不是教别人突破限制,我们只是用学习的眼光看问题

大家都知道ILDasm是。NET程序的反编译工具,它是由Microsoft提供的反编译工具。

它可以直接把。NET程序反编译为IL文件及资源文件,这样即可以非常容易的让黑客进行修改,删除强命名,修改注册码算法等等。。。并且Ilasm再次编译,得到一个正确的,可发布的程序集
并且,这个功能是其它反编译器所不能替代的功能,因为ILDasm真的太重要了。也许有的朋友能理解,有的朋友不能理解,但没关系,我们今天的重点并不是这个。

这是一件不可思议的事,让软件没有了安全保障。幸亏有了XenoCode,它有一项功能即是 Anti ILDasm 。这是一个非常棒的功能,但它是怎么做到的呢?

其实,这是ILDasm的一个限制,当你在程序中制造某个标志后,那么程序集将不可以再被ILDasm反编译,ILDasm会提示您,这个程序集已经是一个被有版权的程序集,您不可能对其反编译。呵呵,我们最可爱的ILDasm会罢工?

我曾经找过相关资料,不过没有找到有什么最简单的方法让自己的程序集变成已有版权的程序集,XenoCode有这样做,不过我不想去分析它。如果有知道的朋友,请与我交流一下

那么我们今天的任务是什么呢?就是把罢工的ILDasm拉回来,让它继续为我们工作。

OK,分析一下吧:经过短暂的分析,让我出了一身汗。。。这样的版权保护有还不如无,一定误导了很多朋友。

为什么我这样说呢?因为我发现,想让ILDasm再次工作,比想象中的简单很多,这根本就档不住任何东西。请看下面我跟踪的代码:

10042B4BD E8 8AFCFDFF CALL ildasmCr.0040B14C //报错函数
2
3
400415FC6 E8 12470100 CALL ildasm.0042A6DD //进入的主函数
5
6
70042AABD FF51 0C CALL DWORD PTR DS:[ECX+C] //判断函数
80042AAC0 3BF7 CMP ESI,EDI
90042AAC2 75 0F JNZ SHORT ildasm.0042AAD3 //关键跳转地址,改为JMP即可
100042AAC4 68 96010000 PUSH 196
110042AAC9 E8 8964FFFF CALL ildasm.00420F57
120042AACE E9 E3090000 JMP ildasm.0042B4B6
13

这已经很明显了,ILDasm只用了一个标志去阻止"已有版权"的程序集,而您只需要修改一个机器指令就可以畅通无阻的反编译任何程序集,并修改其内容再次编译

2.
ildasm.exe的/adv参数

From MSDN Magazine 2001.05 的 BugSlayer 专栏

ildasm /adv assembly.dll

添加/adv参数后,菜单中会多出几项,适用于.NET Framework 1.0 and 1.1。.NET Framework 2.0下好像所有的菜单都直接显示出来了。

3.
Net环境下的程序破解
发信人:windcbf
From: http://www.pediy.com/bbshtml/BBS5/pediy50489.htm
http://www.pediy.com/bbshtml/BBS5/pediy50488.htm

首先介绍一下.Net环境下的程序形成过程。

(1)程序员用高级OO语言(C#,VB.Net,VC.Net,Cobol,Python..)编写代码,这些代码经过
.Net编译器(比如Visual Studio.Net),编译成统一的MSIL(Microsoft Intermediate Language)。NET应用程序是以MSIL的形式出现的,只有在程序执行的时候才通过即时编译器JIT(Just-In-Time)被编译为本机代码。
(2)程序执行的时候,由JIT装入程序的MSIL,JIT同时作了很多其他的工作(装载相应的运行库,安全检测。。) 最后JIT将转化成本地机器码(EXE或者DLL)。
(3)本地机器码被装入内存,开始执行。

就是通过这几个步骤,.net实现了编程语言无关(都转化成MSIL);平台无关(执行的
时候根据本地配置,生成相应的机器码);安全性(JIT里面做了很多检测)。


作为Cracker,可以用.Net Framework SDK自带的工具进行MSIL和EXE(DLL)之间的互换:
利用FrameworkSDK\Bin\目录下的ilasm和ildasm
(1)ilasm示例:
下面的命令对 MSIL 文件 myTestFile.il 进行汇编并产生可执行文件 myTestFile.exe。
ilasm myTestFile
下面的命令对 MSIL 文件 myTestFile.il 进行汇编并产生 .dll 文件 myTestFile.dll。
ilasm myTestFile /dll
下面的命令对 MSIL 文件 myTestFile.il 进行汇编并产生 .dll 文件 myNewTestFile.dll。
ilasm myTestFile /dll /output:myNewTestFile.dll
(2)ildasm示例
下面的命令使 PE 文件 MyHello.exe 的元数据和反汇编代码显示在 Ildasm.exe 的默认 GUI 中。
ildasm myHello.exe
下面的命令对 MyFile.exe 文件进行反汇编,并将结果 MSIL 汇编程程序文本存储在 MyFile.il 文件中。

ildasm MyFile.exe /output:MyFile.il
下面的命令对 MyFile.exe 文件进行反汇编,并将结果 MSIL 汇编程序文本显示到控制台窗口中。

ildasm MyFile.exe /text
如果文件 MyApp.exe 包含嵌入的托管和非托管资源,则下面的命令将产生以下 4 个文件:MyApp.il、MyApp.res、Icons.resources 和 Message.resources:

ildasm MyApp.exe /output:MyApp.il
下面的命令对 MyFile.exe 的 MyClass 类中的 MyMethod 方法进行反汇编,并将输出显示到控制台窗口中。

ildasm /item:MyClass::MyMethod MyFile.exe /text
在上面的示例中,可能有几个具有不同签名的 MyMethod 方法。下面的命令对返回类型为 void 且带有参数 int32 和 System.string 的 MyMethod 方法进行反汇编。

ildasm /item:"MyClass::MyMethod(void(int32,class System.String))" MyFile.exe /text

通过上面分析可知,我们(Cracker)需要分析的对象就是MSIL形式的文件了

>>>MSIL: 从Hello World开始

.Net下的汇编语言(MSIL)看起来要友好的多,学编程都从Hello World开始,这里我们
同样也从一个小小的Hello World开始来逐步了解MSIL,呵呵

所需要的工具仅仅记事本就可以了,打开Notepad,输入代码:
// file:Hello.il “//“都是注释,可省去
// Author:WinHack[CCG]

//定义Assembly(EXE,DLL)
.assembly helloworld //要产生的Assembly(EXE,DLL)名称
{
.ver 1:0:0:0 //版本
}

//用.method定义方法(函数)
.method public static void main() il managed
{
.entrypoint //这里定义整个程序的入口点!
.maxstack 1 //函数所保留的栈槽数量

//导入字符串
ldstr "Hello World!!!\n"

//在控制台(Console)显示
call void [mscorlib]System.Console::WriteLine(class System.String)

ret //返回
}

将上面的文件保存成hello.il文件,打开console界面,用ilasm编译成exe。
D:\>ilasm hello.il
Microsoft (R) .NET Framework IL Assembler. Version 1.0.3705.0
Copyright (C) Microsoft Corporation 1998-2001. All rights reserved.
Assembling 'hello.il' , no listing file, to EXE --> 'hello.EXE'
Source file is ANSI

Assembled global method main
Creating PE file

Emitting members:
Global Methods: 1;
Writing PE file
Operation completed successfully

现在就可以运行程序了!
D:\>hello
Hello World!!!

这就是一个简单得不能在简单的hello world程序了,但是通过这个程序,我想大家对MSIL已经有一个大体上的印象了吧,当然.net framework是一个极其庞大的系统,MSIL里面包含了很多复杂的元素,后面我会尽力逐个介绍,不过好几天没有干活了,导师催得紧,过几天再写吧。有兴趣的发信交流,呵呵^_^

4.
http://www.remotesoft.com/salamander/index.html

Salamander is a .NET decompiler that converts executable files (.EXE or .DLL) from Intermediate Language (IL, MSIL, CIL) binary format to high-level source codes, such as C#, managed C++, Visual Basic.NET, etc. For more than 8,000 classes that have been tested, Salamander always produces equivalent and recompilable codes that are remarkably close to the original source codes.

5.
http://www.remotesoft.com/dotexplorer/

Remotesoft .NET Explorer is a generic object browser and MSIL disassembler with professional look and feel. It offers the same functionality as Microsoft ILDASM disassembler utility, plus low level viewing of metadata and PE format. Remotesoft .NET Explorer works together with our decompiler and obfuscator, and acts as a console for easy navigation and powerful code editing and printing. This tool can be used as a source code editor, and it has a powerful syntax coloring system that recognizes many popular source files, including IL, C#, C/C++, VB, ASP, JAVA, HTML, FORTRAN, PHP, etc.

6.
http://www.aisto.com/roeder/dotnet/

Reflector is a class browser for .NET components. It supports assembly and namespace views, type and member search, XML documentation, call and callee graphs, IL, Visual Basic, Delphi and C# decompiler, dependency trees, base type and derived type hierarchies and resource viewers.

Resourcer is an editor for .resources binaries and .resX XML file formats used with the .NET platform. The program allows the integration of bitmaps, icons and text strings into resource packages. Resourcer allows editing of name/string pairs, import of various kinds of data formats (bitmaps, icons, etc) and merging of resources from different sources.

7.
http://test.saurik.net/anakrino/

8.
MaxtoCode
http://www.cnblogs.com/aiasted/

9.
让Reflector的反编译不能正常工作

By midea0978
From http://www.cnblogs.com/midea0978/articles/110365.html

用过Reflector的都知道,这是一个功能非常强大的反编译工具,几乎能够将你的.net代码完全还原,并且能够立即重新
编译运行,本文提供了一个方法让Reflector的反编译不能正常工作,据研究,目前很多代码混淆工具已经使用了该技术,
为了便于理解,我将该方法姑且叫做代码顺序扰乱技术。
下面来研究一个具体的例子:
1)StrConv.cs源代码如下:

using System;
public class StrTool
{
public static void Main(String[] argv){
String s1="\ud7d1\udec3\ue5b9\ueca6";
int num=0x4308d77e;
Console.WriteLine(conv(s1,num));
}

public static string conv(string str, int num)
{
char[] chArray1 = str.ToCharArray();
for (int num1 = 0; num1 < chArray1.Length; num1++)
{
chArray1[num1] = (char)(chArray1[num1] - num);
num += 0x6fd;
}
return new string(chArray1);
}

}

运行csc strconv.cs编译运行,结果是SHA1,程序的功能是简单的字符串加密处理。
用Reflector打开strconv.exe文件,发现所有代码可以准确的反编译为c#。

2)用ildasm反编译,导出IL文件StrConv.il,打开找到conv方法的代码如下:
.method public hidebysig static string conv(string str,
int32 num) cil managed
{
// 代码大小 65 (0x41)
.maxstack 4
.locals init (char[] V_0,
int32 V_1,
string V_2)
IL_0000: ldarg.0
IL_0001: callvirt instance char[] [mscorlib]System.String::ToCharArray()
IL_0006: stloc.0
IL_0007: ldc.i4.0
IL_0008: stloc.1
IL_0009: br.s IL_0013
IL_000b: ldloc.0
IL_000c: ldloc.1
IL_000d: br IL_0027
IL_0012: stloc.1
IL_0013: ldloc.1
IL_0014: ldloc.0
IL_0015: ldlen
IL_0016: conv.i4
IL_0017: blt.s IL_000b
IL_0019: ldloc.0
IL_001a: newobj instance void [mscorlib]System.String::.ctor(char[])
IL_001f: stloc.2
IL_0020: br.s IL_003f
IL_0022: br IL_003f
IL_0027: ldloc.0
IL_0028: ldloc.1
IL_0029: ldelem.u2
IL_002a: ldarg.1
IL_002b: sub
IL_002c: conv.u2
IL_002d: stelem.i2
IL_002e: ldarg.1
IL_002f: ldc.i4 0x6fd
IL_0034: add
IL_0035: starg.s num
IL_0037: ldloc.1
IL_0038: ldc.i4.1
IL_0039: add
IL_003a: br IL_0012
IL_003f: ldloc.2
IL_0040: ret
} // end of method StrTool::conv

下面将上面的代码顺序扰乱编排如下,中间夹了几个跳转指令br
.method public hidebysig static string
conv(string str,
int32 num) cil managed
{
// 代码大小 50 (0x32)
.maxstack 4
.locals init (char[] V_0,
int32 V_1,
string V_2)
IL_0000: ldarg.0
IL_0001: callvirt instance char[] [mscorlib]System.String::ToCharArray()
IL_0006: stloc.0
IL_0007: ldc.i4.0
IL_0008: stloc.1
IL_0009: br.s IL_0021

IL_000b: ldloc.0
IL_000c: ldloc.1
//开始跳转
br IL_000d


IL_0020: stloc.1
IL_0021: ldloc.1
IL_0022: ldloc.0
IL_0023: ldlen
IL_0024: conv.i4
IL_0025: blt.s IL_000b

IL_0027: ldloc.0
IL_0028: newobj instance void [mscorlib]System.String::.ctor(char[])
IL_002d: stloc.2
IL_002e: br.s IL_0030
br IL_0030

//***移动过来的代码
IL_000d: ldloc.0
IL_000e: ldloc.1
IL_000f: ldelem.u2
IL_0010: ldarg.1
IL_0011: sub
IL_0012: conv.u2
IL_0013: stelem.i2
IL_0014: ldarg.1
IL_0015: ldc.i4 0x6fd
IL_001a: add
IL_001b: starg.s num
IL_001d: ldloc.1
IL_001e: ldc.i4.1
IL_001f: add
//***返回
br IL_0020

IL_0030: ldloc.2
IL_0031: ret
} // end of method StrTool::conv

修改完毕,保存。大家可以看到只是简单的将代码顺序移动了一下,没有其他任何改变。
用ilasm重新组装ilasm strconv.il成新的strconv.exe

3)重新运行结果正常。然后用Reflector打开strconv.exe文件,反编译conv方法为C#,这时就出现错误了。
System.InvalidOperationException: Invalid branching statement for condition expression with target offset 0027.
at _127._4(Int32 )
at _127._1(Int32 )
at _127._1(Int32 , Int32 )
at _127._1(IMethodDeclaration , IMethodBody )
at _123.VisitMethodDeclaration(IMethodDeclaration value)
at _146._1(Boolean )
不信大家可以试试!

本文只是简单演示了这种代码处理的方法,如果多次移动代码,那么要想还原就更加复杂了。
对于这种问题,Reflector在将来的版本看会不会解决,不过目前最新版本仍然不能处理该问题。

(按:简单的跳转方法而已。矛与盾的攻防正未有穷期)

Sunday, April 24, 2005

 

Some notes on Fully Trusted Code and Reflection

1.
作者:Flier Lu
出处:http://flier_lu.blogone.net/?id=1601113

Keith Brown在4月份的MSDN杂志上发表了一篇讨论.NET下安全性的文章,Beware of Fully Trusted Code,其中详细讨论了让 Managed Code 允许在 Fully Trusted 模式下的危害。真是不看不知道,呵呵,现在我是深深感到了 Fully Trusted Code 的可怕。

对一个类型的私有成员变量来说,其私有性的保护实际上只是编译期的

以下为引用:

class DiskQuota {
private:
long MinBytes;
long MaxBytes;
};

以上类型的私有成员变量实际上可以简单的通过unsafe代码中的指针操作访问

以下为引用:

void EvilCode(DiskQuota* pdq) {
// use pointer arithmetic to index
// into the object wherever we like!
((long*)pdq)[1] = MAX_LONG;
}

好在CLR能够通过类型系统完整性检测,一定程度上解决这类有意或缓冲区溢出的问题,除非在程序中显式的要求CLR关闭此类检测。如

以下为引用:

// evilcode.cs
using System.Security.Permissions;

[assembly: SecurityPermission(
SecurityAction.RequestMinimum,
Flags=SecurityPermissionFlag.SkipVerification)]

// your evil code goes here

麻烦的是一旦使用/unsafe参数编译C#程序,或者使用Managed C++编写程序,这个权限就被隐式地设置为最低,以保障代码地正确执行。虽然新版本的C#编译器将提供参数选项打开此权限,但最根本的解决方法还是得通过.NET安全策略来完成。可惜在缺省状态下,本地硬盘上的Managed程序都是在 Fully Trusted 模式下运行的。
可以通过控制面板的管理工具中的"Microsft .NET Framework 1.1配置"工具,在 我的电脑\运行库安全策略\计算机\代码组\All_Code 上通过 编辑代码组属性 看到当前的安全策略设置。安全策略一般分为企业、计算机和用户三级,每级又可根据一定的代码组条件分为多个代码组,设置不同的权限集。
我们接下来看看 Fully Trusted 权限的“威力”所在吧 :P

首先是所谓的私有方法,众所周知它们是可以通过Reflection被直接调用的,呵呵,如下:

以下为引用:

using System;
using System.Reflection;

class EvilCodeWithFullTrust
{
static void CallPrivateMethod(object o, string methodName) {
Type t = o.GetType();
MethodInfo mi = t.GetMethod(methodName,
BindingFlags.NonPublic |
BindingFlags.Instance);
mi.Invoke(o, null);
}
static void Main() {
CallPrivateMethod(new NuclearReactor(), "Meltdown";
}
}

比较好的解决方法是通过检测调用者是否被信任,来保障私有函数不被滥用,如使用StrongNameIdentityPermission属性限制调用者所在assembly必须有相同的public key,可以参见我以前的一篇 BLog 文章《关于信任粒度的讨论》

以下为引用:

public class NuclearReactor {
[StrongNameIdentityPermission(
SecurityAction.LinkDemand,
PublicKey="002400000..."]
private void Meltdown() {
// calling assembly must have specified public key!
}
}

这样可以禁止直接通过Reflection访问私有成员函数或变量。但如果调用者代码具有 Fully Trusted 权限的话,它可以直接使用命令行工具 caspol -s off 或者通过代码,完全关闭 StrongNameIdentityPermission 进行的检测,呵呵

以下为引用:

using System.Security;
class EvilCodeWithFullTrust {
static void Main() {
SecurityManager.SecurityEnabled = false;
// now call Meltdown via reflection!
}
}

改变SecurityManager.SecurityEnabled的值需要SecurityPermissionFlag.ControlPolicy权限,对应于配置中安全性下的允许策略控制权限, Fully Trusted 情况下是打开的

既然系统自动的调用者验证可以被跳过,是否能通过程序手工进行验证呢?以下代码在调用不可完全信任的插件之前,通过降低权限来限定插件的能力,可以说这是一种非常好的安全编程习惯。

以下为引用:

using System.Security;
using System.Security.Permissions;

class WellMeaningCode {
public void CallPlugIn(EvilCode plugin) {
// put a CAS modifier on the stack that denies all file system access
new FileIOPermission(
PermissionState.Unrestricted).Deny();
plugin.DoWork();
CodeAccessPermission.RevertDeny();
}
}

但是在 Fully Trusted 权限下还是无能为力,直接对PermissionSet.Assert函数的调用使得权限检查再一次被跳过。

以下为引用:

class EvilCodeWithFullTrust {
void DoWork() {
new PermissionSet(
PermissionState.Unrestricted).Assert();
// happily access the file system
// regardless of the caller''s deny!
}
}

前面提到配置工具中对安全策略的配置有企业、机器和用户三层,实际上还有AppDomain这一层,可以通过AppDomain.SetAppDomainPolicy载入单独的安全策略限定其后需要执行的代码,ASP.NET就是通过这种方法限定执行代码的权限。但是具有 Fully Trusted 权限的代码可以打破 AppDomain 边界的限制,并可以通过调用其他 Unmanaged Code 实现对其他 AppDomain 侵入。好在可以通过修改 machine.config 文件让 ASP.NET 运行在较低的权限集中,如

以下为引用:

configuration>
system.web>
securityPolicy>
trustLevel name="Full" policyFile="internal" />
trustLevel name="High" policyFile="web_hightrust.config" />
trustLevel name="Medium" policyFile="web_mediumtrust.config" />
trustLevel name="Low" policyFile="web_lowtrust.config" />
trustLevel name="Minimal" policyFile="web_minimaltrust.config" />
/securityPolicy>
!-- level="[Full|High|Medium|Low|Minimal]" -->
trust level='Medium'/>
/system.web>
/configuration>

但因为很多程序员没有耐心完成最小权限集的调整工作,导致大部分情况下为了省事将缺省权限设置为 Fully Trusted,导致上述各种精心设计的安全检测都形同虚设。再好的技术如果没有相应的管理来保障,只是摆设而已。
解决这个问题,一方面需要程序员对其代码的权限做限定和调整,另一方面需要开发厂商提供静态和动态的权限分析工具辅助,最后还得配合传统的NT权限设置来把关,毕竟CLR还是运行在NT的用户的权限集下。

有兴趣的朋友可以进一步阅读这篇文章:Writing managed code for semi-trusted environment

此外 Brown 还是 Programming Windows Security 一书的作者,他在书中详细介绍了 Windows 环境下的安全子系统的运行机制和使用方法,并一定程度上涉及到了网络认证、COM+和Internet环境下的安全性问题,非常详尽值得一读。此书已经由电力出版社翻译出版 《Windows安全性编程》。

2.
Writing managed code for semi-trusted environment

By (c) Ivan Medvedev 2003
From http://www.dotnetthis.com/Articles/WritingForSEE.htm

Say what?

In .NET’s CLR Microsoft has introduced the concept of semi-trusted code – a code that can do some things and can not do other things, for example – read files but not the registry. Well, the concept was not exactly new J, but the .NET implementation, IMO, got it right and it is actually very usable. In 3 words the CLR security system works like this:

- A loader loads an assembly

o It collects information about the assembly called the “evidence”. There are 5 major types of evidence that built into the system – Url, Zone, Site, StrongName, Publisher; evidence system is extendable and there could be custom types of evidence, too.

o Once the evidence it gathered it is fed into the security policy system, which maps a set of evidence to a set of permissions.

o That set of permissions is called an “assembly grant set”, and it is assigned to each assembly when it is loaded.

- A protected resources is accessed – e.g. a file open operation is attempted

o The FileStream class constructor creates an object of FileIOPermission describing what file and how it being accessed and does .Demand() on it.

o This initiates an operation called a “stack walk”. Stack walk verifies that every method on the stack has the permission to open the file, by looking at the method’s assembly grant set.

o If any of the methods on the stack belongs to an assembly whose grant set does not include FileIOPermission, a SecurityException is thrown. This prevents something called “luring attacks” (do I remind you of Dr. Evil?), where lower trusted code orchestrates highly trusted code into doing bad things.

o There are things called “stack walk modifiers” – Assert(), Deny() and PermitOnly(). They make the security system much more flexible, but also make it easier for a trusted library writer to screw up.



If you had to write your own FileStream class you would have to use an Assert() in addition to a Demand(). The Assert() would stop a check for the permission to call into unmanaged code (which you will have to call because you need Win32 APIs to do files) – you wouldn’t want your callers to necessarily have that permission, you only want them to have the permission to work with files.



If you haven’t worked with code access security before the above is probably confusing (it took me about a year to fully and deeply understand this stuff J). If you want to know more about it, there are a couple of good books – “.NET Framework Security” by Brian LaMacchia et al, “Writing Secure Code 2nd edition” by Michael Howard (look me up in the acknowledgements section J), also “Visual Basic .NET Code Security Handbook” by Eric Lippert is great for VB.NET people.

Also, Greg Fee has recently written a good article on stack walk in his blog.

The Problem

Anyway, semi-trusted environment is when your application does not get a full set of permissions that would allow it to do everything. If you look at the default security policy (run “.NET Framework Configuration” from the control panel or “caspol –l” if you are a hardcode command line guy), you can kind of see that everything coming from zone MyComputer has FullTrust (can do everything) and stuff coming from Internet gets a permission set named “Internet” (we will look at it in detail). Assemblies coming from Intranet or network shares get LocalIntranet permission set – that is why you get a security exception when running your favorite application (whose creator didn’t read this article) off a mapped drive (read Shawn’s blog on how you can fix their mess J).



If you have looked at the default Internet permission set in caspol tool output you have seen a bunch of xml goo that represents it. I will try to translate it for you. So, as an application coming from the Internet (in v1.1 of the Framework), you are entitled to:

- Open files the user has explicitly pointed you to (FileDialogPermission – Open);

- Use up to 10240 bytes of Isolated Storage (a cool feature, I will probably write another entry just about that in the future), with isolation by application (no cross app talking) – IsolatedStorageFilePermission;

- You can actually execute code (didn’t expect it? J) – SecurityPermission – Execution;

- You can display safe top level windows - you won’t be able to fully control the window title, it is going to say “Internet” and then the site name – UIPermission - SafeTopLevelWindows;

- You can use your own application clipboard (no passing stuff between applications) – UIPermission – OwnClipboard

- You can print (only with user’s explicit approval) – PrintingPermission – SafePrinting;

- You can read web contents originating from the same location as your application - Same Site Web.



Not much, is it? Now, if we take LocalIntranet, it adds the following:

- You can read ‘USERNAME’ environment variable (EnvironmentPermission);

- You can read and save files with explicit user’s approval (FileDialogPermission);

- You can use all the IsolatedStorage space you want, and you can use isolation by assembly – applications can share data this way (IsolatedStorageFilePermission);

- You can dynamically create assemblies in memory (ReflectionPermission – ReflectionEmit);

- You can redirect assembly bindings (SecurityPermission – BindingRedirects);

- You can assert permissions (use .Assert() method) (SecurityPermission – Assertion);

- You can display any kind of windows with full control over them and you can fully use the clipboard (UIPermission – Unrestricted);

- You can use DNS services (DnsPermission);

- You can print with the default printing settings without user’s explicit approval (PrintingPermission – DefaultPrinting);

- You can access Event Log (EventLogPermission);

- You can read files from the directory of your origin and its subdirectories, and you can enumerate files/folders there (Same directory FileIO - Read, PathDiscovery).





Now, how in the world I can write code that works with these restrictions?

Well, that’s the hard part. First there are some things that make your code un-runnable in default semi-trusted environment for Internet or Intranet right away:

- Unverifiable code: C# code with unsafe blocks or compiled with /unsafe option (you will have it, for example, if you are using pointers), any C++ code, code written in IL that uses unverifiable constructs (see ECMA specs);

- Corrupted code or code changed by some obfuscators that made it unverifiable;

- Code that has declarative assembly permission requests for more trust than the security policy gives them (you can see assembly requests with permview.exe tool);

- Code that calls into unmanaged code (by using native COM or DllImport attribute);

- Code using classes/methods that require permissions outside of the security sandbox (for example Reflection, some file i/o, Registry, etc.). The documentation that comes with the Framework tries to document the security requirements for all APIs, however you may here and there stumble upon a method where the doc does not list its real requirements.



(Note that we are talking about the default security policy here. Remember that you can always adjust policy on the client machine to give more permissions to a certain assembly).



It is in fact very hard to figure out whether your code is going to work off the Internet or Intranet at the development stage without trying it out. In the next release of the .NET Framework we are working very hard to make this analysis possible. Meanwhile, trying and failing is the most reliable method. To make it easier you can use something called “assembly permission requests”. You can put a declarative attribute like the following in your assembly to ensure that no more permissions are granted to it, even if it is loaded from the local drive – this will help you catch sandbox violations easier:



[assembly: FileIOPermissionAttribute(SecurityAction.RequestOptional, Unrestricted=true)]

[assembly: SecurityPermissionAttribute(SecurityAction.RequestOptional, Execution=true)]



The attribute above will ensure that your assembly only gets FilIOPermission and the permission to execute. It is really a good practice to have those assembly requests on your assemblies in general; by using them you are limiting your liability – imagine your library has a security hole that would allow attackers to make it do bad things you didn’t intend it to do – having a self-restriction like this may help to avoid such situation.



I am sure you will get a sense of what’s allowed and what is not pretty soon.



I have done something you can call a case study of this problem by trying to implement a useful tool that would work in Internet or Intranet environment. I have created an assembly metadata parser and IL disassembler (you can think of it as a mini-version of ILDASM), along with a simple managed code obfuscator. You can find them at http://www.dotnetthis/Samples/mangler.htm, and I encourage you to check them out because they are really cool. So, as you can see the exercise was successful and I got some nice apps out of it and some knowledge to share with you.



Some problems I hit while developing these applications and their solutions.

Writing a metadata parser / IL disassembler is a problem in itself, and doing it in partially-trusted code is even harder. First, you can not use Reflection (not that it helped much anyway) or unmanaged metadata reading APIs (like IMetaDataImport), so you have to parse the PE files from scratch. Then, you can not use pointers to walk through the memory and pull bytes. Then, when you found the data you are interested in you can not just map it to a struct and be done (in fully trusted code you could use something like Marshal.PtrToStructure).



So let me give you a few specific examples.



Reading files from disk

In Internet/Intranet sandbox you can not just open a file and read its contents – you will get a security exception. However if you look closer at the set of allowed permissions you can see a FileDialogPermission there. This basically means that you can display an OpenFileDialog, and if the user selects a file, you can grab an already opened Stream object off of it. The code goes like this:



// create an OpenFileDialog instance

OpenFileDialog openFileDialog = new OpenFileDialog();

// see if the user picked a file

if (openFileDialog.ShowDialog() != DialogResult.OK)

return;

// check if the file actually exists

if (!openFileDialog.CheckFileExists)

return;

// open the file for read and grab the stream

FileStream fs = (FileStream)openFileDialog.OpenFile();

// read the contects into a byte array

byte[] array = new byte[fs.Length];

fs.Read(array, 0, (int)fs.Length);



This will get you the contents of a file the user has picked.



Saving a file to disk

Saving file to disk is accomplished approximately the same way, except you use a SaveFileDialog. Intranet zone, by default, gets the permission to use SaveFileDialog. Internet zone however, does not. That is why in my sample obfuscator, if you are running it off the Internet, you won’t be able to save the obfuscated files (you will be able to view them though) unless you adjust the policy to give my application something less restrictive, for example LocalIntranet permission set. Another thing about SaveFileDialog is that with the default policy you are not going to get permissions to control the dialog caption, so in case you have to save multiple files, you won’t be able to say “Saving file X” in the title. As you can see, in my sample I am using the main control window’s status bar for it. This restriction is done to prevent evil people from confusing you by trustworthy-looking window captions.



Calling into your other components

If your application consists of several assemblies it is important that it is able to load them and run them. Both Internet and Intranet zone in the default v1.1 policy give assemblies the permission to connect back to the server of origin to fetch components – this is needed for loading referenced libraries. If you are on v1.0, your Internet zone may get not permission at all and you will have to grant some permissions to it manually, including the permission to read things from the server if your application has multiple assemblies.

Another important thing that I mentioned is being able to run. If your assembly is strong name signed and you call into it from your partially trusted application this will not work unless you have AllowPartiallyTrustedCallersAttribute on it:



[assembly: AllowPartiallyTrustedCallersAttribute]





Working with binary data

As I already mentioned, parsing metadata is a lot of work involving walking binary data structures. In plain C I would just cast a pointer to byte to a pointer to a structure describing a certain metadata header and I have easy access to the fields. In unsafe managed code I would use Marshal class to do essentially the same. However in security sandboxed codes I don’t have that luxury and I need to invent my own way to make it easier. What I have come up with in my application is a pseudo-pointer class that makes it easier for me to emulate pointer operation C-style. It goes something like this (some error checking and other parts are omitted for better clarity):



public struct SafePointer

{

internal byte[] _array;

internal int _index;



// ...



public static implicit operator byte(SafePointer pp)

{

return pp._array[pp._index];

}



public static explicit operator UInt16(SafePointer sp)

{

return (ushort)(((byte)(sp+1)<<8) | (byte)(sp));

}



// ... other conversion skipped



public static SafePointer operator +(SafePointer pp, int n)

{

return new SafePointer(pp._array, pp._index+n);

}



// ... skipped



public static SafePointer operator ++(SafePointer sp)

{

sp._index++;

return sp;

}



// ... the rest of operators and methods skipped

}



A class like this allows a very convenient pointer operations, for example:



SafePointer spBase = new SafePointer(array, 0);



SafePointer spTemp = spBase + offset;

while(spTemp < spEnd)

{

Console.WriteLine((UInt16)spTemp);

spTemp+=2;

}



This code will read 2 bytes words from a block of memory and print them.



I then created a universal Header class that, given information about the fields, such as their names, offsets and length, would accept a SafePointer and let you read the values of the fields based either on the field name or its index. It was then very easy to subclass that class and come up with customized header classes for reading various kinds of headers living in the metadata. Same deal for metadata tables and so forth.

This gave me a very powerful and flexible design which allowed many different things. If you saw a MetaInfo sample in the Tools Developers Guide, I wrote a similar tool using the library I have created, and the whole tool was less than 2 screens of code.



Checking your permissions

When writing for semi-trust, it is important for the application to be aware of the permissions it has, so that the user will not get an ugly security exception dialog. For example, I mentioned above that the default policy gives permissions to display a SaveFileDialog to Intranet, but not to Internet. If you try to invoke this functionality while working off the Internet, a SecurityException will be thrown. A more graceful way to deal with this kind of situation is checking your permissions. To do that you should create an instance of the permission class that describes the resource you are trying to access and then you have two options:

- Call its Demand() methods and prepare to catch a SecurityException. If you caught it that means you (or your callers) do not have the permission.

- Do SecurityManager.IsGranted(…). This will tell you if the permission is granted to your assembly.

There is an important difference between the two. The first approach initiates a stack walk and will check all the callers; it imitates what happens when you actually use the resource and is more accurate. The second approach will check your assembly’s permissions, but not its callers’. If you are working within the default security policy the 2nd approach is good – it is fast and it will give you the right result. The 1st approach you will need if you adjust the security policy to give your assembly more trust, in this case you want to check all the callers to make sure the thing will actually work (and if it does not you may want to use Assert() to stop the stack walk in your assembly).

Here is the code I used to check for save file dialog permission:



FileDialogPermission fdp = new FileDialogPermission(FileDialogPermissionAccess.Save);

if (!SecurityManager.IsGranted(fdp))

{

MessageBox.Show("Saving files is not allowed"

return;

}

// do stuff

Getting the performance right

Writing fast semi-trusted code is a big problem in itself. When you need to process large amounts of data and are unable to use unsafe code your performance is going to be affected. First of all, you will likely have to use boxing/unboxing (casting to/from Object), which is expensive. Then, you won’t be able to use some processor/memory-friendly optimization. To give you an example, I have looked at two different functions comparing byte arrays. The first function works in semi-trust and goes through the arrays byte by byte. The second function uses unsafe code blocks, pointers and casts 8-byte chunks of data to UInt64, which are then compared. The second function is more than 3 times faster than the 1st. Finally, when you run FullTrust, there is a lot of performance optimizations in the security system that skip security checks when possible. In semi-trusted environment such as Internet or Intranet much less optimizations are possible and the security checks that are happening when you use some APIs will potentially take more time. Don’t try to turn the security off though, because if you do, next time you visit my obfuscator sample I will tell it to obfuscate your whole disk and nothing is going to stop it! J

3.
作者:jobs
出处:http://www.blogcn.com/user9/szujobs/index.html?dd=2004-6-6

如何调用只有私有构造函数的类

当我试用ObjectSpaces时,ObjectSpaces竟然能够调用只有私有构造函数的类。例如:
以下内容为程序代码:

Class A
{
  private A() {}
}

ObjectSpaces能够创建A的实例,我刚看到的时候,吃了一惊,呵呵…… 后来,借助Reflector分析整理学会了此技巧。

你不能通过Reflection直接创建只有私有构造函数的类,但是你可以通过一些偏门技巧绕过此限制,亦可谓之奇淫技巧。何为奇淫技巧?
其大概思路这样的:
以下内容为程序代码:

Type[] paramTypeArray = new Type[] {};
Type rtnType = ...;
String methodName = ...;
bool skipVisibility = true;
Module module = ...;
DynamicMethod method = new DynamicMethod(
    methodName,
    rtnType, //returnType
    paramTypeArray, //parameterTypes
    module,
    skipVisibility
    );

ILGenerator ilGen = method.GetILGenerator();
ilGen.Emit(OpCodes.Newobj, ctor);

以上是思路,我们也来写一段代码,让其能够调用缺省构造函数创建对象实例,具体代码:

第一步, 定义一个Delegate:
以下内容为程序代码:

public delegate T CreateInstanceDelegate();

第二步,定义构建Delegate的方法,关键在此:
以下内容为程序代码:

private CreateInstanceDelegate BuildDelegate()
{
  Type type = typeof(T);

  AssemblyName assemblyName = new AssemblyName();
  assemblyName.Name = "System.Data.ObjectSpaces.Dynamic";

  AssemblyBuilder assemblyBuilder
    = AppDomain.CurrentDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.RunAndSave);

  Module module
    = assemblyBuilder.DefineDynamicModule("WebData", "DynamicAssembly.dll", true);

  Type[] paramTypeArray = new Type[] {};
  Type rtnType = type;
  String methodName = "call_privateCtor";
  bool skipVisibility = true;

  DynamicMethod method = new DynamicMethod(
      methodName,
      rtnType,
      paramTypeArray,
      module,
      skipVisibility
      );

  ConstructorInfo ctor = type.GetConstructor(
        BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance,
        null,
        new Type[0],
        null
        );

  ILGenerator ilGen = method.GetILGenerator();
  ilGen.Emit(OpCodes.Newobj, ctor);
  ilGen.Emit(OpCodes.Ret);

  return (CreateInstanceDelegate ) method.CreateDelegate(typeof(CreateInstanceDelegate ));
}

第三步,定义创建实例的方法:
以下内容为程序代码:

public T CreateInstance()
{
  CreateInstanceDelegate createInstDelegate = BuildDelegate();
  return createInstDelegate();
}

第四步,如此使用:
定义一个私有缺省构造函数的类
以下内容为程序代码:

class A
{
  private A() {}
}

创建实例的代码:
以下内容为程序代码:

A a = CreateInstance();
Console.WriteLine("create A instance" );

如此奇淫技巧,就能够绕过可见性访问限制构造对象。

4.
发信人: flier (小海 [寻找风车中]), 信区: DotNET
标 题: Re: 如何调用只有私有构造函数的类
发信站: BBS 水木清华站 (Mon Jun 7 21:27:53 2004), 站内

昨天和jobs讨论了一下,ms这么做应该是为了提高调用时的效率
如果只是要完成功能的话,用refection足以,Full Trusted 代码是无敌的 :P

using System;
using System.Reflection;

namespace PrivCon
{
public class Demo
{
string _name;

private Demo(string name)
{
_name = name;
}

private void SayHello()
{
Console.WriteLine("Hello " + _name);
}
}

class EntryPoint
{
[STAThread]
static void Main(string[] args)
{
ConstructorInfo con = typeof(Demo).GetConstructor(
BindingFlags.Instance | BindingFlags.NonPublic, null,
new Type[] { typeof(string) }, null);

if(con != null)
{
Console.WriteLine("call " + con.ToString());

object obj = con.Invoke(new object[] { "Flier Lu" });

MethodInfo method = typeof(Demo).GetMethod("SayHello",
BindingFlags.Instance | BindingFlags.NonPublic,
null, new Type[0], null);

if(method != null)
{
Console.WriteLine("call " + method.ToString());

method.Invoke(obj, new object[0]);
}
}
}
}
}

5.
作者:hush
出处:http://www.cnblogs.com/hush/archive/2004/04/29/8064.aspx

今天在微软新闻组里,看到有人问对于以下这个类:

class Test
{
public Test()
{
throw new Exception("Can not use this constructor";
}
public void Hello()
{
Console.WriteLine("hello World";
}
}
如何成功调用它的Hello方法。

这个类的构造函数里会扔出异常,而类实例化的时候会自动调用构造函数,这样用普通的方法我们永远也得不到一个可以成功调用方法的类的实例。那么有什么办法,可以突破构造函数这个限制?

用反射似乎不行,因为反射要生成一个实例,也还是要调用构造函数,那么有什么其它方法吗?

呵呵,请原谅我卖了那么长时间的关子,好东西总要留到最后,不是吗? 

其实,我们可以这样做:

class main
{
static void Main(string[] args)
{
//此处获得了一个未经初始化的实例(没有调用构造函数)
Test t = (Test)System.Runtime.Serialization.FormatterServices.GetUninitializedObject( typeof(Test) ;
t.Hello();
}
}

是不是有点奇技淫巧的感觉? 嘿嘿

我第一次看到这个方法是在思归呓语上的一个帖子(在思归的blog上找了半天,也没找到原帖,还是用goolge才找到了....)

btw,思归的帖子: http://blog.joycode.com/saucer/posts/407.aspx

6.
发信人: hBifTs (Programing?Reverse:Crack...), 信区: DotNET
标 题: FormatterServices 好用的东东..
发信站: BBS 水木清华站 (Mon May 24 19:32:39 2004), 站内


原文: http://www.cnblogs.com/hbifts/archive/2004/05/23/11035.aspx

在前面一个博客园的hush的文章如何不调用构造函数而获得一个类的实例!中,我们知道了使用System.Runtime.Serialization.FormatterServices.GetUninitializedObject可以得到一个没有调用构造函数的对象..

在得到这个对象后,那么如何在不继续调用构造函数的情况下面对其Private成员/其父类的Private成员进行赋值呢??

呵呵,你可能会说,使用Reflection.对,Reflection是可以得到Private Field..
好,给你下面的一个代码.你能否通过Reflection,给Father类中的objFather赋值?
class Father{
private object objFather;
public Father( object obj){
objFather = obj;
}

public object GetFather(){
return objFather;
}
}

class Son : Father{
private object objSon ;
public Son( object objSon,object objFather) : base(objFather){
this.objSon = objSon;
}
}
object obj1 = new object();
object obj2 = new object();
Son x = new Son( obj1,obj2);
Son t = (Son)System.Runtime.Serialization.FormatterServices.GetUninitializedObject( typeof(Son) ;
......//你的代码.
t.GetFather() == obj2
即,你通过什么方式让上面的等式成立??



呵呵..
Reflection不行吧..
这里,我们如果要想完成上面的任务,就得使用FormatterServices这个类了~
通过MSDN,我们知道他还有这两个函数 一个,GetObjectData,得到一个Object里的所有的Field的值.
PopulateObjectMembers,给一个Object中的所有的Field赋值..

哈哈.好了,就用这两个函数.
代码如下:
FieldInfo[] fields = typeof( Father).GetFields(BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance);
object[] objfields = FormatterServices.GetObjectData(x,fields);
t= FormatterServices.PopulateObjectMembers(t,fields,objfields);

到这里,上面的等式就可以成立了:P


这里面其实是通过Serialize函数来完成的. 所谓的GetObjectData应该就是相当于把这个对象 序列化,这样就可以得到其中的Field的值了.然后再通过反序列化的手段 PopulateObjectMembers(...) 把所得到的数据再放到我们自己的对象中~

--

just for fun~~~~

山自高兮水自深!當塵霧消散,唯事實留傳

Welcome to My Blog We Miss you~ : http://www.cnblogs.com/hBifTs


※ 来源:·BBS 水木清华站 smth.org·[FROM: 218.197.209.*]

补充:
还是把思归老大的帖子也贴一下

在.NET中,怎么生成一个只有私有(private)构造函数的类的对象?譬如象这样的类

class PrivateClass
{
public string Name;
public int Age;
private PrivateClass()
{
Name = "not initialized";
Age = 0;
}
}

先想一想,再看一个答案


using System;
using System.Reflection;

Type t = typeof(PrivateClass);
ConstructorInfo[] cis = t.GetConstructors(BindingFlags.Instance | BindingFlags.NonPublic);
PrivateClass ptest = (PrivateClass)cis[0].Invoke(null);
Console.WriteLine("{0} {1}", ptest.Name, ptest.Age);

另一个答案
作者:Scott Hanselman (按:这家伙也是CP上的老人啦)
出处:http://www.hanselman.com/blog/PermaLink.aspx?guid=410

using System;
namespace Corillian.Testing
{
class PrivateClass
{
public string Name;
public int Age;
private PrivateClass()
{
Name = "not initialized";
Age = 0;
}
}

class Test
{
static void Main(string[] args)
{
/// The following statement will not work as the constructor is private
/// PrivateClass newpTest = new PrivateClass();
/// But you can create the object through Serialization
PrivateClass ptest = (PrivateClass)System.Runtime.Serialization.FormatterServices.GetUninitializedObject( typeof(PrivateClass) ;
ptest.Name = "Scott";
ptest.Age = 0x1D;

Console.WriteLine( String.Format("{0} {1}",ptest.Name,ptest.Age ;
}
}
}
6/10/2004修改

补充:
flier老大指出

btw:用Delegate.InternalAlloc也可以完成相同功能
static public object InternalAlloc(Type type)
{
foreach(MethodInfo method in
typeof(Delegate).GetMethods(BindingFlags.NonPublic | BindingFlags.Static))
{
if(method.Name == "InternalAlloc"
{
return method.Invoke(null, new object[] { type });
}
}
throw new Exception("static method InternalAlloc is not a member of class Delegate";
}

(Test)InternalAlloc(typeof(Test));
6/11/2004修改

Saturday, April 23, 2005

 

Lock, critical section and threadsafe in DotNet

1.
C# 中 lock 关键字的实现
By flier

刚刚在这篇文章 《How is lock keyword of C# implemented? 》http://blogs.msdn.com/junfeng/archive/2004/02/18/75454.aspx 中看到MS内部关于C#的lock关键字实现的一个讨论。

以下为引用:

Subject: RE: How is lock keyword of C# implemented?

At the core, it’s typically one ?lock cmpxchg“ instruction (for x86) for entry, and one for exit, plus a couple dozen other instructions, all in user mode. The lock prefix is replaced with a nop on uniprocessor machines.

The “lock cmpxchg” instruction basically stores the locking thread’s id in the object header, so another thread that tries to lock the same object can see that it’s already locked.

The actual implementation is a lot more complicated, of course – we use the object header for other purposes, for example, so this must be detected and dealt with, plus when a thread leaves the lock, we must detect whether other threads are waiting and so on…

Thanks

回想起前两天分析过的临界区实现,就顺便看了看rotor这方面的实现代码,发现和Windows中临界区的实现思路基本上相同。
在rotor中,每个引用对象内部实现是一个Object对象(sscliclrsrcvmobject.h:126)的实例。而对象同步机制的实现,则是通过和Object对象绑定的ObjHeader对象(sscliclrsrcvmsyncblk.h:539)中的SyncBlock结构完成的。这种实现思路跟Delphi中的VMT的实现很相似,rotor中Object对象指针的-4偏移处存储绑定的ObjHeader对象,Delphi则在负偏移处保存VMT表。

以下为引用:

class Object
{
//...

// Access the ObjHeader which is at a negative offset on the object (because of
// cache lines)
ObjHeader *GetHeader()
{
return ((ObjHeader *) this) - 1;
}

// retrieve or allocate a sync block for this object
SyncBlock *GetSyncBlock()
{
return GetHeader()->GetSyncBlock();
}

//...
};

ObjHeader::GetSyncBlock(syncblk.cpp:1206)方法从缓冲区获取或者创建新的SyncBlock对象。SyncBlock对象则是一个使用lazily created策略的可缓存结构,调用其Monitor完成对象的实际锁定工作。

以下为引用:

// this is a lazily created additional block for an object which contains
// synchronzation information and other "kitchen sink" data

class SyncBlock
{
//...

AwareLock m_Monitor; // the actual monitor

void EnterMonitor()
{
m_Monitor.Enter();
}

//...
};

AwareLock类型是一个很类似临界区的轻量级同步对象,其Enter(syncblk.cpp:1413)方法使用FastInterlockCompareExchange函数尝试锁定此Monitor。如果无法锁定则判断此Monitor的所有者线程是否是当前线程:是则调用线程嵌套锁定函数;否则等待此对象锁定状态的改变。

以下为引用:

Thread *pCurThread = GetThread();

for (; {

// Read existing lock state.
LONG state = m_MonitorHeld;

if (state == 0) {

// Common case: lock not held, no waiters. Attempt to acquire lock by
// switching lock bit.
if (FastInterlockCompareExchange((LONG*)&m_MonitorHeld, 1, 0) == 0)
break;

} else {

// It's possible to get here with waiters but no lock held, but in this
// case a signal is about to be fired which will wake up a waiter. So
// for fairness sake we should wait too.
// Check first for recursive lock attempts on the same thread.
if (m_HoldingThread == pCurThread)
goto Recursion;

// Attempt to increment this count of waiters then goto contention
// handling code.
if (FastInterlockCompareExchange((LONG*)&m_MonitorHeld, state + 2, state) == state)
goto MustWait;
}

}

可以看到这儿的实现思路和临界区的实现基本上相同。
FastInterlockCompareExchange函数(util.hpp:66)则是MS那个讨论里面提到的lock cmpxchg指令的调用之处。此函数根据编译时选项,被替换成CompareExchangeUP/CompareExchangeMP两个函数分别处理单/多处理器情况。可以参考vmi386cgenx86.cpp中的InitFastInterlockOps函数(cgenx86.cpp:2106)实现。在386平台上,这两个函数完全由汇编语言实现(i386asmhelpers.asm:366, 440)。

以下为引用:

CmpXchgOps FastInterlockCompareExchange = (CmpXchgOps)CompareExchangeUP;

// Adjust the generic interlocked operations for any platform specific ones we
// might have.
void InitFastInterlockOps()
{
_ASSERTE(g_SystemInfo.dwNumberOfProcessors != 0);

if (g_SystemInfo.dwNumberOfProcessors != 1)
{
//...

FastInterlockCompareExchange = (CmpXchgOps)CompareExchangeMP;

//...
}
}

以下为引用:

FASTCALL_FUNC CompareExchangeUP,12
_ASSERT_ALIGNED_4_X86 ecx
mov eax, [esp+4] ; Comparand
cmpxchg [ecx], edx
retn 4 ; result in EAX
FASTCALL_ENDFUNC CompareExchangeUP

FASTCALL_FUNC CompareExchangeMP,12
_ASSERT_ALIGNED_4_X86 ecx
mov eax, [esp+4] ; Comparand
lock cmpxchg [ecx], edx
retn 4 ; result in EAX
FASTCALL_ENDFUNC CompareExchangeMP

值得注意的是那个讨论里面提到“The lock prefix is replaced with a nop on uniprocessor machines”,据rain的分析,NT核心部分的DLL也做了类似的优化。

2.
切勿锁定类型对象

MTT翻译组翻译

http://blog.vckbase.com/song/articles/444.aspx

为什么使用 Lock(typeof(ClassName)) 或 SyncLock GetType(ClassName) 是错误的

最近,Microsoft .NET 运行库的性能设计师及资深 Microsoft 开发人员 Rico Mariani 在一封电子邮件中与 GUI 博士进行了交流,其中提到的一种相当普遍的做法(遗憾的是,这种做法在我们的一些文档中也曾提到过,虽然我们将进行修改)实际上却存在着很大的问题。他询问 GUI 博士能否帮忙发布消息,告诉程序员不应该采用这种做法。博士当然很乐意帮忙。

这种非常普遍的做法是什么呢?其实就是对类型对象加锁。在 C# 中,加锁的做法是 lock(typeof(ClassName)),其中,ClassName 是某个类的名称;在 Microsoft Visual Basic .NET 中,加锁的做法是 SyncLock GetType(ClassName)。

背景知识:在多线程编程中,lock/SyncLock 语句用于创建代码中一次只执行一个线程的关键部分或简要部分。(如果您需要同时更新对象中的多个字段,则可能需要该语句 — 您希望确保其他线程不会同时尝试更新该对象!)此语句将锁定与您指定的对象相关联的唯一监视对象,如果其他线程已经锁定了该监视对象,则等待。一旦它锁定了监视对象,任何其他线程都无法锁定该监视对象,除非您的线程解除锁定,解除锁定会在封闭块的结尾自动发生。一种常见的用法是锁定 this/Me 引用,这样,只有您的线程可以修改您在使用的对象 — 不过,更好的做法是锁定您即将修改的特定对象。锁定尽可能小的对象的好处是可以避免不必要的等待。

GetType 和 typeof 返回对该类型的类型对象的引用。System.Type 类型的类型对象包含使您能够反映类型的方法,这意味着您可以找到它的字段和方法,甚至可以访问字段和调用方法。一旦您拥有对类型对象的引用,就可以创建该对象的一个实例(并且,如果您使用 Type.GetType shared/static 方法,就可以按名称获得对类型对象的引用)。

因此,类型对象非常方便。但是,有些程序员喜欢“滥用”这种方式,借此来代替可以对其进行加锁的 static/Shared 对象。(遗憾的是,我们在 C# 文档和 Visual Basic .NET 文档中都提到了这种方法,暗示这是一种建议采用的做法。)在这种情况下,这些文档中的建议是错误的(我们会进行纠正)。这种做法是不 可接受的,更不用说建议采用了。

原因是这样的:由于一个类的所有实例都只有一个类型对象,因此从表面看,锁定类型对象相当于锁定类中包含的静态对象。只要您锁定类的所有实例,等到其他线程访问完任一实例的任何部分,然后锁定访问,这样您就可以安全地访问静态成员,而不会受到其他线程的干扰。

这种做法的确有效,至少在大多数情况下是这样的。但它也有一些问题:首先,获得类型对象实际上是一个很缓慢的过程(尽管大多数程序员会认为这个过程非常快);其次,任何类中的其他线程、甚至在同一个应用程序域中运行的其他程序都可以访问该类型对象,因此,它们就有可能代替您锁定类型对象,完全阻止您的执行,从而导致您挂起。

这里的基本问题是,您并未拥有该类型对象,并且您不知道还有谁可以访问它。总的来说,依靠锁定不是由您创建、并且您不知道还有谁可以访问的对象是一种很不好的做法。这样做很容易导致死锁。最安全的方式就是只锁定私有对象。

但除此之外,还有更严重的问题。由于在当前版本的 .NET 运行库中,类型对象有时会在应用程序域之间(但不是在进程之间)共享。(通常这没有问题,因为它们是不变的。)这意味着,运行在其他应用程序域(但在同一进程)中的另一个应用程序有可能对您要锁定的类型对象进行加锁,并且始终不释放该类型对象,从而使您的应用程序发生死锁。并且,这样可以很容易地获得类型对象的访问权限,因为该对象具有名称 — 该类型的完全限定名!请记住,lock/SyncLock 会一直阻塞(这是挂起的含蓄说法),直到它可以获得锁定为止。很显然,依靠锁定其他程序或组件可以锁定的对象不是一种很好的做法,并且会导致死锁。

即使该类型对象在您的应用程序域中是唯一的,这仍然是一种不好的做法,因为任何代码都可以访问公共类型的类型对象,从而导致死锁的发生。如果您在应用程序中使用的组件不是您编写的,这种做法尤其成问题。(即使是 lock(this)/SyncLock Me 也可能有这个问题,因为其他人可能会锁定您。即使发生了这种事情,问题的根源也可能会比锁定类型对象而导致的死锁更容易发现,因为您的对象并不是跨应用程序域的全局可用对象。)

那么,应该采用什么方法呢?非常简单:只要声明并创建一个对象作为锁,然后使用它而不是 类型对象来进行锁定。通常,为了复制问题代码的语义,您会希望此对象是 static/Shared — 当然,它其实应该是私有的!总之,您可以将以下问题代码:

// C#
lock(typeof(Foo)) { // BAD CODE! NO! NO! NO!
// statements;
}

' VB .NET
SyncLock GetType(MyClass) ' BAD CODE! NO! NO! NO!
' statements
End SyncLock

更改为以下正确代码:

// C#
lock(somePrivateStaticObject) { // Good code!
// statements;
}

' VB .NET
SyncLock GetType(somePrivateStaticObject) ' Good code!
' statements
End SyncLock

当然,您必须已经拥有一个要锁定的私有静态对象(如果您使用锁定来修改静态对象,实际上您可能已经有了一个!)或者必须创建一个。(使它成为私有对象可以避免其他类锁定您的对象。)请不要尝试锁定不是引用(对象)类型的字段,例如 int/Integer。那样会出现编译器错误。如果您没有要锁定的私有静态对象,可能需要创建一个哑对象:

// C#
Class MyClass {
private static Object somePrivateStaticObject = new Object();
// methods of class go here--can lock somePrivateStaticObject
}

' VB .NET
Class MyClass
Private Shared somePrivateStaticObject As New Object
' methods of class go here--can lock somePrivateStaticObject
End Class

您需要单独分析每种情况,以确保不会出现问题,但通常上述技巧会奏效。

有两点需要注意:首先,类以外的任何代码都无法锁定 MyClass.somePrivateStaticObject,因此避免了许多死锁的可能。由于死锁属于那种最难找到根源的问题,因此,避免发生死锁的可能是一件很好的事情。

其次,您知道,您的应用程序中只有一份 MyClass.somePrivateStaticObject 的副本,并且系统上运行的其他每个应用程序也只有一个副本。因此,在同一个应用程序域中的应用程序之间没有相互影响。GUI 博士希望您能明白为什么修改后的代码比原来的问题代码更加可靠和强大。

总之,不要锁定类型对象,因为您并不知道哪里又出现问题了。锁定类型对象的过程很慢,并且可能发生死锁情况。这是一种很不好的编程习惯。相反,您应该在对象中锁定静态对象。

致谢!

GUI 博士在此感谢 .NET 运行库的性能设计师 Rico Mariani 提供了这方面的宝贵意见。

3.
安全的线程同步
原作:Jeffrey Richter [msdn.2003.01]
翻译:刘未鹏
出处:http://blog.csdn.net/pongba/archive/2004/08/24/83918.aspx

(按:再次复习2004/3/11-12的概念)

到目前为止,线程同步最为普遍的用途是确保多线程对共享资源的互斥访问。对于同步单个进程中的多个线程以实现互斥访问,Win32 API中的CRITICAL_SECTION结构以及与它相关的函数提供了最为快速和高效的途径,Microsoft .NET Framework并没有提供CRITICAL_SECTION结构,但提供了一个类似的机制——该机制通过System.Threading.Monitor类和SyncBlock得以实现。

在这个专栏里,我会解释.NET Framework如何支持线程同步的这个普遍用途。另外,我还要解释SyncBlock和Monitor被设计成现在这个样子的动机以及它们是如何工作的。最后我还要解释为什么这个设计是糟糕的,以及如何用正确和安全的方式去使用该机制。



绝妙的主意



.NET Framework采用了OOP式的结构。这就意味着:开发者构造对象,然后调用类型的成员来操纵它。然而有时候这些对象也会被多个线程所操纵,因此,为了确保对象的状态不被破坏,必须进行线程同步。在设计.NET Framework时,Microsoft的设计师们决定创造一个新的机制来让开发者们轻易地同步一个对象。

基本想法是这样的:堆上的每个对象都关联着一个可以被用于线程同步的数据结构(非常类似于Win32的CRITICAL_SECTION)。然后,FCL(framework class library,框架类库)再提供一些方法(method)——当你将对象的引用传递过去时——使用这个数据结构来同步线程。

如果将这个设计运用在Win32下的非托管的C++类上,则该类看起来像这样:



图1. A CRITICAL_SECTION for Every Object

class SomeType {

private:

// 为每个对象关联一个私有的CRITICAL_SECTION字段

CRITICAL_SECTION m_csObject;



public:

SomeType() {

// 在构造函数中初始化CRITICAL_SECTION字段

InitializeCriticalSection(&m_csObject);

}



~SomeType() {

// 在析构函数中delete CRITICAL_SECTION字段

DeleteCriticalSection(&m_csObject);

}



void SomeMethod() {

// 在该函数中,我们使用了对象的CRITICAL_SECTION字段

// 来同步多个线程对该对象的访问

EnterCriticalSection(&m_csObject);

// 在这里可以执行线程安全的代码了...

LeaveCriticalSection(&m_csObject);

}



void AnotherMethod() {

// 在该函数中,我们使用了对象的CRITICAL_SECTION字段

// 来同步多个线程对该对象的访问

EnterCriticalSection(&m_csObject);

// 在这里可以执行线程安全的代码了...

LeaveCriticalSection(&m_csObject);

}

};



本质上,.NET Framework为每个对象关联一个类似CRITICAL_SECTION的字段,并且负责对它进行初始化和删除。开发者要做的唯一一件事情是:在需要线程同步的方法中加入一些代码以进入(Enter)和离开(Leave)该字段。



实现绝妙的主意



现在,很明显,为堆上的每个对象都关联一个CRITICAL_SECTION字段是一件很浪费的事情,特别是由于大多数对象从不需要线程安全的访问。因此,.NET Framework小组设计了一个更高效的途径来提供前面描述的功能。下面就来说说它是如何工作的:

当CLR(公共语言运行时)初始化时,它分配一个SyncBlock的缓存区。一个SyncBlock就是一个可以根据需要关联到任何对象的内存块。SyncBlock具有与Win32 CRITICAL_SECTION相同的字段。

堆上的每个对象在创建时都关联了两个额外的字段——第一个是MethodTablePointer(方法表指针),包含了该类型的方法表的地址。基本上,这个指针使你能够获取堆上的每个对象的类型信息。事实上,当你调用System.Object的GetType方法时,该方法通过对象的MethodTablePointer字段来确定对象的类型。另一个额外的字段——称为SyncBlockIndex——包含一个32位的有符号整数,在SyncBlock缓存区中索引一个SyncBlock。

当一个对象被构造时,它的SyncBlockIndex字段会被初始化为一个负值——表示根本不指向任何SyncBlock。然后,当某个方法被调用以同步对该对象的访问时,CLR在SyncBlock缓存区中寻找一个空闲的SyncBlock,并且让该对象的SyncBlockIndex字段指向它。换句话说,SyncBlock只在某个对象需要同步字段时才会被关联到该对象。而当不再有任何线程对该对象进行同步访问时,该对象的SyncBlockIndex字段就会被重置为一个负值,并且相应的SyncBlock也被释放,以便在将来可以被重新关联到其它对象。

图2是这个思想的具体表现形式。在该图的CLR Data Structures部分,你可以看到,系统所知道的每个类型都有一个对应的数据结构。你也可以看到SyncBlock结构的集合。在该图的Managed Heap部分你可以看到有三个对象(ObjectA,ObjectB,ObjectC)被创建了。每个对象的MethodTablePointer字段都指向相应类型的方法表。通过方法表,你可以获得每个对象的类型。所以,我们可以很容易的看到,ObjectA和ObjectB都是SomeType类型的实例,而ObjectC是AnotherType类型的实例。

ObjectA的SyncBlockIndex字段被设为0,这表示SyncBlock #0目前正被ObjectA使用。另一方面,ObjectB的SyncBlockIndex被设为-1,表示并没有任何SyncBlock与它关联。最后,ObjectC的SyncBlockIndex为2,表明它正在使用SyncBlock #2。在本例中,SyncBlock #1没有被使用,但是将来可能会与某个对象关联。

因此,从逻辑上说,堆上的每个对象都关联着一个SyncBlock——它可以被用于快速互斥的线程同步。然而,从物理上说,SyncBlock仅当被需要时才会与某个对象关联,并且在对象不再需要它时从对象上脱离。这意味着内存的使用是有效率的。另外,如果有必要,SyncBlock缓存可以创建更多的SyncBlock,因此你不用担心它会由于同一时刻同步对象过多而被用尽。



使用Monitor来操纵SyncBlock


既然你已经理解了SyncBlock,就让我们来看看如何锁定一个对象。要锁定或解锁一个对象,你可以使用System.Threading.Monitor类。该类型的所有方法都是静态的。调用下面的方法可以锁定一个对象:



public static void Enter(object obj);



当你调用Enter方法时,它首先检查指定的对象的SyncBlockIndex是否为负值,如果是的,则该方法从SyncBlock缓存中找一个空闲的SyncBlock并且将它的索引保存到对象的SyncBlockIndex中。一旦对象上已经关联了SyncBlock,则Enter方法就会检查对象的SyncBlock,看看当前是否有另一个线程正拥有这个SyncBlock。如果当前没有任何进程拥有它,则当前调用Enter的线程成为该对象的所有者。另一方面,如果另一个线程已经拥有了该SyncBlock,则当前调用Enter的线程会挂起,直到那个线程释放对SyncBlock的所有权。

如果你想写更具防御性的代码,则可以调用下面的TryEnter方法之一:



public static Boolean TryEnter(object obj);



public static Boolean TryEnter(object obj,

int millisecondsTimeout);



public static Boolean TryEnter(object obj,

TimeSpan timeout);



第一个TryEnter只是简单的检查调用它的线程能否获得对象的SyncBlock的所有权,如果成功则返回true。另外两个方法则允许你指定一个等待时间,表示你允许线程等待所有权多久。如果不能获得所有权,则三个方法都会返回false。

一旦获得了所有权,代码就可以安全地访问对象的字段了。访问完毕之后,线程还应该调用Exit来释放SyncBlock:



public static void Exit(object obj);



如果线程并没有获得指定对象的SyncBlock的所有权,则调用Exit会抛出一个SynchronizationLockException。同时还要注意,线程可以递归式获取一个SyncBlock的所有权,每次成功的Enter必须对应一个Exit,这样最终SyncBlock才会被释放(这一点与Win32的CRITICAL_SECTION是一样的——译者)。



Synchronizing the Microsoft Way



现在让我们看看图3的示例代码,它展示了如何使用Monitor的Enter和Exit方法来锁定和解锁一个对象。注意,LastTransaction属性(property)的实现需要调用Enter,Exit以及一个临时变量dt。这是非常重要的,这样可以避免返回一个已经被破坏的值——如果一个线程调用PerformTransaction时另一个线程在访问该属性就会发生这种情况。



用C#的Lock语句来简化代码



因为这种“调用Enter——访问受保护资源——调用Exit”的模式是如此普遍,所以C#干脆提供了特殊的语法来简化这种代码。图4的两段C#代码片断功能相同,但是后者更为简洁。使用C#的lock语句,你可以将Transaction类充分简化。特别要注意图5中所展示的改进后的LastTransaction属性,它不再需要临时变量了。

除了能够简化代码外,lock语句还确保了Monitor.Exit一定会被调用,从而即使在try块中产生了一个异常,SyncBlock也会被释放。你应该始终将异常处理与同步机制结合起来使用,以确保锁被正确释放。然而,如果你使用C#的lock语句,则编译器会自动帮你生成正确的代码。另外,Visual Basic .NET也有一个类似于C#的lock语句的SyncLock语句,它们做同样的事情。



Synchronizing Static Members the Microsoft Way



Transaction类示范了如何同步访问一个对象的实例字段。但是,如果你的类型定义了一些静态字段以及访问这些字段的静态函数又当如何呢?在这种情况下,堆上并没有该类型的实例,因而也就没有可用的SyncBlock或者传递给Monitor.Enter和Monitor.Exit方法的对象引用。

事实上,包含某个类型的类型描述符的内存块是位于堆上的对象。图2中并没有表现出这一点,但是SomeType的类型描述符和AnotherType的类型描述符所占的内存块其实本身都是对象,并且同样也有MethodTablePointer字段和SyncBlockIndex字段。这就意味着SyncBlock可以被关联到一个类型,并且类型对象(type object,指“描述一个类型”的对象)的引用可以被传递给Monitor的Enter和Exit方法。在图6所示的Transaction类中,所有的成员都改成了静态的,并且PerformTransaction方法和LastTransaction属性也作了改动以展示Microsoft希望开发者如何同步对静态成员的访问。

在PerformTransaction方法和LastTransaction属性中,你不再会看到this关键字,因为在静态成员中不能使用它。我将类型的类型描述符对象的引用传给lock语句。这个引用是通过C#的typeof操作符得到的,typeof操作符返回指定对象的对象描述符的引用。在Visual Basic .NET中,GetType操作符具有同样的功能。



为什么绝妙的主意并不那么绝妙



如你所见,使堆上的每个对象逻辑上关联一个用于同步的数据结构的主意听起来很不错。但是实际上这是一个糟糕的主意。听我解释原因。还记得在本文开头展示的非托管的C++代码吗?如果由你来写,你会将CRITICAL_SECTION字段的访问权限设为public吗?当然不会——那简直是荒谬的。将这个字段设为public会允许程序中的任何代码操纵该CRITICAL_SECTION结构,这样恶意代码很容易就能够将使用该类型的实例的任何线程死锁住。

呃...猜猜看发生了什么——SyncBlock正如一个public字段一样!任何一段代码在任何时候都可以将任何对象的引用传给Monitor的Enter和Exit方法。事实上,任何类型描述符的引用也同样可以被传给Monitor的方法。

图7的代码显示了这种情况是多么糟糕。这里,Main方法中创建了一个App对象,然后锁定该对象,并在某个时刻发生一次垃圾收集(在这段代码中,强制一次垃圾收集),当App的Finalize方法被调用时,它也会试图锁定该对象,但是由于程序的主线程已经锁定了该对象,所以CLR的Finalize线程无法锁定它。这就导致CLR的Finalize线程停止了——于是在当前进程(可能包含多个AppDomain)中再也不会有其它对象可以被finalize,也不再有其它可finalize的对象在堆上的内存会被回收。

幸运的是有一个解决方案,只不过那意味着你得抛开Microsoft的设计和建议。取而代之的是定义一个private的System.Object字段作为你的类型的成员,构造它,然后将其引用传递给C#的lock语句或Visual Basic .NET的SyncLock语句。图8展示了如何重写Transaction类以便让用于同步的对象成为该类的私有成员。同样地,图9展示了当Transaction类的成员全为静态时如何去重写它。

看起来,仅仅为了同步而构造一个System.Object对象是比较怪异的。我感觉Microsoft对Monitor类的设计并不恰当。应该让你为每个想要同步的类型(原文为type,疑为object)构造一个Monitor类型的实例。这样,静态方法(Monitor类的静态方法)就会成为不需要System.Object参数的实例方法。这将解决所有的问题,并且充分简化开发者的编程模型。

另外,如果你创建具有许多字段的复杂类型,则在任何时候,你的方法和属性可能只需要锁定这些字段的一个子集。你始终可以通过将指定字段的引用传给lock或Monitor.Enter来锁定它。当然,如果字段为私有(我始终建议如此),则我只会考虑这样做。如果你想要将几个字段一起锁定,那么你可以始终将其中的一个传给lock或Enter。或者,你还可以构造一个System.Object对象——它的唯一意图就是用于锁定一集字段。lock段(临界段)越细化,代码的性能和可测性就越好[1]。

[1] 译注:作者的意思是,应该保持lock(临界)段的短小,换句话说,一个lock(临界)段应该执行尽量少的代码,这样才能保证其它线程在lock(临界)区上的等待时间尽量短,并且死锁的可能性也更小。



未装箱的(unboxed)值类型实例



在结束这个专栏之前,我想要指出一个有关同步的bug,我第一次遇到它时花了好几个小时来跟踪。下面的代码片断示范了这个问题:



class AnotherType {



// 一个 未装箱的(unboxed) Boolean 值类型实例

private Boolean flag = false;


public Boolean Flag {

set {

Monitor.Enter(flag); // 将flag装箱并锁定装箱后的对象

flag = value; // 而实际的值却未受保护

Monitor.Exit(flag); // 将flag装箱并试图unlock装箱后的对象

}

}

}



你可能会惊讶于在这段代码中并没有发生任何线程同步!原因是:flag是个未装箱的值类型实例,而并非一个引用类型。未装箱的值类型实例并没有MethodTablePointer和SyncBlockIndex这两个额外的字段。这就意味着一个未装箱的值类型实例不可能有一个与它关联的SyncBlock。

Monitor的Enter和Exit方法要求一个指向堆上的对象的引用。当C#,Visual Basic .NET和许多其它编译器看到一段代码试图将未装箱的值类型实例传给需要对象引用的方法时,它们会自动生成代码来将该值类型的实例装箱(box)。装箱后的实例(位于堆上)将会拥有一个MethodTablePointer和一个SyncBlockIndex,因而可以被用于线程同步。然而,每调用这样的函数一次就会进行一次新的装箱,即产生一个新的装箱后的实例,这个实例与以前装箱的实例都不相同,也就是说,我们每次lock和unlock的都是不同的对象。

例如,在上面代码片断中,当Flag属性的set方法被调用时,它调用了Monitor的Enter方法。Enter需要一个引用类型,因此flag被装箱,并且装箱后的对象的引用被传递给Enter,该对象的SyncBlock现在归调用线程所有。如果另一个线程现在也要访问这个属性,那么flag将会被再次装箱,产生一个新的对象,它拥有自己的SyncBlock。另外,对Exit的调用也会导致一次装箱操作。

正如我前面所说,我花了好几个小时才发现问题所在。如果你想要同步对一个未装箱的值类型实例的访问,那么你必须分配一个System.Object对象,并利用它来进行同步。图10中的代码是正确的。

另外,如果你使用C#的lock语句来代替Monitor.Enter与Monitor.Exit,那么C#编译器会帮你避免意外地试图去lock一个值类型。当你将一个未装箱的值类型实例传给lock语句时,C#编译器会报错。例如,如果你试图将一个Boolean(C#中的bool)传给lock语句,那么你将看到如下的错误:“error CS0185:'bool' is not a reference type as required by the lock statement”。而在Visual Basic .NET中,如果你对SyncLock语句使用未装箱的值类型实例,编译器也会报错:“error BC30582: 'SyncLock' operand cannot be of type 'Boolean' because 'Boolean' is not a reference type”。


图 3. Using Enter and Exit Methods

class Transaction {



// Private field holding the time of

// the last transaction performed

private DateTime timeOfLastTransaction;



public void PerformTransaction() {

// Lock this object

Monitor.Enter(this);



// Perform the transaction...



// Record time of the most recent transaction

timeOfLastTransaction = DateTime.Now;



// Unlock this object

Monitor.Exit(this);

}



// Public read-only property returning

// the time of the last transaction

public DateTime LastTransaction {

get {

// Lock this object

Monitor.Enter(this);



// Save the time of the last transaction

// in a temporary variable

DateTime dt = timeOfLastTransaction;



// Unlock this object

Monitor.Exit(this);



// Return the value in the temporary variable

return(dt);

}

}

}



图 4. Regular and Simple Lock and Unlock

// Regular function

public void SomeMethod() {

// Lock the object

Object oTemp = this;

Monitor.Enter(oTemp);

try {

// Access the object

...

// Unlock the object

}

finally {

Monitor.Exit(oTemp);

}

// Return

}



// Simple function

public void SomeMethod() {

// Lock the object

lock (this) {



// Access the object

...

// Unlock the object

}



// Return

}



图 5. Transaction Class

class Transaction {



// Private field holding the time of

// the last transaction performed

private DateTime timeOfLastTransaction;



public void PerformTransaction() {

lock (this) {

// Perform the transaction...



// Record time of the most recent transaction

timeOfLastTransaction = DateTime.Now;

}

}



// Public read-only property returning

// the time of the last transaction

public DateTime LastTransaction {

get {

lock (this) {

// Return the time of the last transaction

return timeOfLastTransaction;

}

}

}

}



图 6. New Transaction Class

class Transaction {



// Private field holding the time of

// the last transaction performed

private static DateTime timeOfLastTransaction;



public static void PerformTransaction() {

lock (typeof(Transaction)) {

// Perform the transaction...



// Record time of the most recent transaction

timeOfLastTransaction = DateTime.Now;

}

}



// Public read-only property returning

// the time of the last transaction

public static DateTime LastTransaction {

get {

lock (typeof(Transaction)) {

// Return the time of the last transaction

return timeOfLastTransaction;

}

}

}

}



图 7. Threads Banging Heads

using System;

using System.Threading;



class App {

static void Main() {

// Construct an instance of the App object

App a = new App();



// This malicious code enters a lock on

// the object but never exits the lock

Monitor.Enter(a);



// For demonstration purposes, let's release the

// root to this object and force a garbage collection

a = null;

GC.Collect();



// For demonstration purposes, wait until all Finalize

// methods have completed their execution - deadlock!

GC.WaitForPendingFinalizers();



// We never get to the line of code below!

Console.WriteLine("Leaving Main");

}



// This is the App type's Finalize method

~App() {

// For demonstration purposes, have the CLR's

// Finalizer thread attempt to lock the object.

// NOTE: Since the Main thread owns the lock,

// the Finalizer thread is deadlocked!

lock (this) {

// Pretend to do something in here...

}

}

}



图 8. Transaction with Private Object

class Transaction {



// Private Object field used

// purely for synchronization

private Object objLock = new Object();



// Private field holding the time of

// the last transaction performed

private DateTime timeOfLastTransaction;



public void PerformTransaction() {

lock (objLock) {

// Perform the transaction...



// Record time of the most recent transaction

timeOfLastTransaction = DateTime.Now;

}

}



// Public read-only property returning

// the time of the last transaction

public DateTime LastTransaction {

get {

lock (objLock) {

// Return the time of the last transaction

return timeOfLastTransaction;

}

}

}

}



图 9. Transaction with Static Members

class Transaction {



// Private, static Object field

// used purely for synchronization

private static Object objLock = new Object();



// Private field holding the time of

// the last transaction performed

private static DateTime timeOfLastTransaction;



public static void PerformTransaction() {

lock (objLock) {

// Perform the transaction...



// Record time of the most recent transaction

timeOfLastTransaction = DateTime.Now;

}

}



// Public read-only property returning

// the time of the last transaction

public static DateTime LastTransaction {

get {

lock (objLock) {

// Return the time of the last transaction

return timeOfLastTransaction;

}

}

}

}



图 10. Now There's Synchronization

class AnotherType {



// An unboxed Boolean value type

private Boolean flag = false;



// A private Object field used to

// synchronize access to the flag field

private Object flagLock = new Object();



public Boolean Flag {

set {

Monitor.Enter(flagLock);

flag = value;

Monitor.Exit(flagLock);

}

}

}

Friday, April 22, 2005

 

Cooperative Fiber Mode Sample

by Dino Viehland
from http://blogs.msdn.com/dinoviehland/

1.
Well it’s been a while since I’ve blogged… There’ve been a few distractions, starting with Beta 1, then vacation, and now things are back to normal. So it seems reasonable to blog about something that we’ve just shipped in Whidbey Beta 1: The cooperative fiber mode sample that’s been included in the SDK. You can get the freely downloadable SDK here. If you load up the documentation, go to the Contents, and select Samples->Application Samples->Cooperative Fiber Mode Application Sample you’ll be able to get to the code.

This sample shows how to write an extremely simple fiber mode host. But it does so with a twist. Instead of scheduling the tasks in a cooperative, automatic, and transparent fashion this sample exposes the concept of fibers to managed code . This allows the managed code author to use fibers as the unmanaged code author would. This results in an implementation that for the most part relies upon native threads. The only time fibers are actually used is when the user explicitly chooses to use them. This closely matches the experience you’d have programming with fibers in the unmanaged world. More sophisticated hosts would want to implement their own synchronization primitives that would schedule other tasks while blocking.

Over the next several blog entries I plan to give a walk through of the new fiber mode sample. During this blog I’ll start with loading the runtime and getting bootstrapped into managed code. But first let me give a brief warning. Fiber mode is complex and hard to get right. You probably don’t want to implement fiber mode into your application on a whim. Weigh your other options carefully before jumping into running in fiber mode.

Loading the runtime hasn’t changed in a significant fashion from v1.0 and v1.1. We start with a call that should be familiar to anyone who has written code to host the runtime before:

// load up the runtime

HRESULT hr = CorBindToRuntimeEx(

NULL, // version of the runtime to request

NULL, // flavor of the runtime to request

0, // runtime startup flags

CLSID_CLRRuntimeHost, // clsid of ICLRRuntimeHost

IID_ICLRRuntimeHost, // IID of ICLRRuntimeHost

(PVOID*)&pClrHost); // a pointer to our punk that we get

The only difference you may notice is that we’re asking for a different interface. Previously we asked for CLSID_CorRuntimeHost and IID_ICorRuntimeHost. In Whidbey we have a brand new interface that let you do much more.

Next, we have a new call:

hr = pClrHost->SetHostControl(static_cast(&hostControl));

This new command is handing off to the runtime our IHostControl interface. It’s not a required call for hosting the runtime, but it’s the mechanism through which we provide our threading managers (so it is required for fiber mode). One callback on this API is GetHostManager which when called by the runtime allows the host to provide a manager. The host managers provide threading, memory, assembly loading, or other low level functionality to the runtime. Throughout the next couple of articles I’ll be strictly focused on the threading managers.

After we hand off our host control we’re back to familiar territory again:

hr = pClrHost->Start();

Now the runtime has been started and we can start executing managed code. And the way we start doing this has changed in Whidbey. There are a few different mechanisms, and I’ll mention 2 of them. The first is the app domain manager. This allows a host to have an assembly loaded into every domain, including the default domain. Once this assembly is loaded the host can start running managed code directly from it. The other option is a sample call to execute an assembly passing it a string of arguments. This sample uses the 2nd method as it’s simpler and sufficient for our purposes:

hr = pClrHost->ExecuteInDefaultAppDomain(

curDir, // directory to assembly

L"Microsoft.Samples.FiberBootstrap",// Type name to load

L"EntryPoint", // Method name to execute

bootstrapArgs, // Arguments to be passed in

&retVal); // return value of function

The executed function exists in a separate managed DLL (FiberBootstrap.dll). It simply parses the string and then calls AppDomain.ExecuteAssembly on the current domain. This allows the sample to run any managed assembly you pass to it. This EXE that gets loaded then can interact with the fiber mode APIs.

At this point we’ve loaded the runtime, loaded an assembly into the default domain and executed managed code. That’s really all there is to getting started. If you were a simpler host you wouldn’t even need the SetHostControl call! In the next article I’ll discuss what that SetHostControl call is setting up for us.

2.
In the last blog entry I went over how we start the runtime. Once it was loaded we were running managed code, but I glossed over the managers that were present and participating in running the system. Today we start off with CHostControl::GetHostManager. CHostControl is CoopFiber’s implementation of IHostControl, and GetHostManager is the only API we implement.

HRESULT __stdcall CHostControl::GetHostManager(

/* [in] */ REFIID riid,

/* [out] */ void **ppObject)

{

if(riid == IID_IHostTaskManager)

{

if(m_pTaskMan == NULL)

{

m_pTaskMan = new CHostTaskManager();

}

*ppObject = static_cast(m_pTaskMan);

m_pTaskMan->AddRef();

return(S_OK);

}

else if(riid == IID_IHostSyncManager)

{

if(m_pTaskMan == NULL)

{

m_pTaskMan = new CHostTaskManager();

}

*ppObject = static_cast(m_pTaskMan);

m_pTaskMan->AddRef();

return(S_OK);

}

return(E_NOINTERFACE);

}

What are we doing here? CHostControl has one task manager. That task manager actually implements both IHostTaskManager and IHostSyncManager. These are the minimal APIs you need to implement fiber mode of any sort. IHostTaskManager provides the mechanisms for creating tasks. IHostSyncManager provides the mechanisms for providing synchronization between those tasks.

At load time, the CLR calls the host-implemented IHostControl interface and asks for the managers that the CLR knows about. The host can respond by handing back an interface pointer or it can respond by returning E_NOINTERFACE. Virtually any combination of managers is supported. The one gotcha is your thread and synchronization must be compatible. The default synchronization implementation in the CLR is not fiber aware. That’s the reason that we implement both a task and synchronization manager for this fiber mode sample.

Here our GetHostManager implementation simply lazily creates the task manager for the first caller and hands it off to the runtime. If it’s a manager the host doesn’t know about we return E_NOINTERFACE.

The host control also provides a couple of other APIs that we’re not using here. The first is SetAppDomainManager. This is where the host is notified of the creation of an app domain. The host is passed both the ID of the newly created app domain and an IUnknown pointer for the app domain manager. Because CoopFiber doesn’t use an app domain manager we’ll never get any calls here. We simply return E_NOTIMPL.

The other API is GetDomainNeutralAssemblies. This is another API that CoopFiber doesn’t use because it doesn’t care whether assemblies are loaded domain neutral or not. Again, it simply returns E_NOTIMPL.

So that’s it for the host control. Stay tuned for the next entry when I’ll dig into our task manager. Things will start to get really interesting then!

3.
When I last left off we went over the host control. The host control hands off our task and synchronization managers. Now it’s time to take a look at the two host-implemented managers, starting with the task manager.

I’m going to skip over the minor details such as constructors, destructors, etc… and only focus in on the details of the APIs that communicate with the CLR. The APIs we care about the most are GetCurrentTask and CreateTask. A fiber mode host with it’s own scheduler would pay more attention to APIs like Sleep (where we merely call the OS API), SwitchToTask (which is a NOP in CoopFiber), and the other APIs. Because CoopFiber doesn’t have a scheduler we aren’t concerned with these APIs. So if we look at GetCurrentTask:

_ASSERTE(pTask);

CHostTask *curTask = CHostTask::GetCurrentTask(false);

if(curTask==NULL)

{

// 0x1E00 - value returned by GetCurrentFiber when not a fiber

if(GetCurrentFiber()==(PVOID)0x1E00)

{

curTask = CHostTask::CreateTask(ConvertThreadToFiber(NULL),TASK_FLAG_RUNNING);

}

else

{

curTask = CHostTask::CreateTask(GetCurrentFiber(),TASK_FLAG_RUNNING);

}

}

curTask->AddRef();

*pTask = (IHostTask *)curTask;

return(S_OK);

We simply check to see if the current task has been created. If it hasn’t we create a new task. Depending on whether or not we’re already running on a fiber (by some chance) we’ll re-use the fiber, otherwise we’ll create a new fiber. Here we choose to allow any random threads that enter into the CLR to be converted to fibers. A host could also deny these threads entrance by returning HOST_E_INVALIDOPERATION.

The other interesting API is CreateTask:

_ASSERTE(ppTask!=NULL);

_ASSERTE(pStartAddress);

CHostTask *tmp = CHostTask::CreateTask();

if (tmp == NULL)

{

_ASSERTE(FALSE && "Out of memory in CreateTask!");

return E_OUTOFMEMORY;

}

tmp->SetStart(pStartAddress, pParameter);

tmp->AddRef();

*ppTask = (IHostTask *)tmp;

return(S_OK);

This isn’t really doing anything too fancy. We’re just creating out internal task implementation which implements IHostTask. We’re handing that off to the runtime. Later the runtime will call Start on it and it’ll kick off the running of a new task. We’ll look at the IHostTask implementation next time.

4.
Last time we left off with the CLR calling into the host to create a task. So far everything’s been very simple. For each interface we’ve only needed to implement a couple of methods to get the bulk of the work done. While I’ve certainly left out a couple of lines of code here and there, nearly every other API in the interface merely returns S_OK.

Now things start to get really interesting! The IHostTask interface has just a few functions which are exposed to the runtime. But there’s a lot of other functionality related to the task embedded in here.

We’ll start off at the natural place to start a task: Start. It’s pretty simple:

FlagSet(TASK_FLAG_NOTSTARTED,false);

_ASSERTE(m_fiberAddr == NULL);

m_fiberAddr = ::CreateFiber(0,(LPFIBER_START_ROUTINE )CHostFiberProc,(PVOID)this);

// create the new thread

::CreateThread(NULL,NULL,reinterpret_cast(::StartNewThread),this,NULL,NULL);

return(S_OK);

We create a fiber for the new task, and we create a new thread. This new thread starts in StartNewThread, defined in callbacks.cpp. It looks something like:

CHostTaskManager *myManager = CHostTask::GetManager();

CHostTask *curTask = static_cast(lpParameter);

LPVOID fiberAddr = ConvertThreadToFiber(NULL);

curTask->SwitchIn();

BOOL fResult = ConvertFiberToThread();

Here we simply convert the new thread over to fibers, and then switch in the fiber passed as the argument. This brings us back to our CHostTask implementation where we need to look at the SwitchIn logic. This is the most complicated piece of code we’ve encountered yet.

CHostTask *curTask = GetCurrentTask();

_ASSERTE(curTask != this);

TlsSetValue(CHostTask::CurTaskTlsIndex,this);

this->AddRef();

// Save our thread handle so we can queue APCs

if(!DuplicateHandle(GetCurrentProcess(),

GetCurrentThread(),

GetCurrentProcess(),

&m_hCurThread, 0 , FALSE,

DUPLICATE_SAME_ACCESS))

{

m_hCurThread = INVALID_HANDLE_VALUE;

_ASSERTE(!"Duplicate handle failed");

}

FlagSet(TASK_FLAG_RUNNING, true);

::SwitchToFiber(GetFiberAddress());

// when the fiber switches back we need to switch in our

// previous task.

if(curTask->m_pCallback!=NULL)

{

HRESULT hr = curTask->m_pCallback->SwitchIn(GetCurrentThread());

_ASSERTE(SUCCEEDED(hr));

}

if(curTask->FlagCheck(TASK_FLAG_EXITING))

{

_ASSERTE(curTask->GetSwitchingTo());

curTask->ExitTask();

curTask->GetSwitchingTo()->SwitchIn();

}

// release the ref for TLS from the previous thread

CloseHandle(m_hCurThread);

m_hCurThread = INVALID_HANDLE_VALUE;

Release();

SetThreadPriority(GetCurrentThread(), curTask->m_iPriority);

SetThreadLocale(curTask->m_lcid);

What’s going on here? We have a couple of things to worry about when switching tasks. We have the task that we’re switching to (this) and we have the task we’re switching from (curTask).

One of the issues we’re concerned with is the lifetime of the task. While it’s running we don’t want it cleaned up, so we hold a reference to it (in Thread Local Store). A more complicated host would probably have a pool of tasks rather than the simple TLS mechanism.

If we need to alert a task we’ll queue an Asynchronous Procedure Call (APC) to it. Therefore the next thing we do is save the current thread’s handle into this task. We’ll use this in IHostTask::Alert. Finally we set the running flag and switch over to the new task.

The interesting thing to note about this method is there are 2 halves to it. After we call SwitchToFiber we are running on a different fiber on a different stack. We’ll only return to the bottom half after someone has switched back to “curTask”. This is the task that we switched away from. When this happens it’s now the bottom half’s job to tell the runtime that “curTask” is once again running.

It’s possible when we get to the bottom half “curTask” was only switched in to exit. If so we’ll notify the runtime and immediately re-run the task we’ve been set to re-run. This is an implementation detail of CoopFiber to allow calling ExitTask on the managed fiber implementation.

We’re nearly done so we clean up the resources we allocated in the top half for “this”. It’s no longer running, so we don’t need a reference to it. Finally we restore the settings for the thread that were stored in the task. These would have been changed when we did the intital SwitchToFiber which either switched in a task that was at the bottom half of SwitchIn, or the top of CHostFiberProc (in callbacks.cpp).

Wow, so that’s how we start a task! We create a new thread, that thread gets converted over to fibers, and we switch in the newly created task (which has a fiber already associated with it). That’ll end up in CHostFiber proc which we’ll cover in a future edition.

There’s just one more detail to cap off the life time of a task, and that is our internal SwitchOut API. All it essentially does is set some state and notify the runtime of the switch out:

FlagSet(TASK_FLAG_RUNNING,false);

if(m_pCallback!=NULL)

{

HRESULT hr = m_pCallback->SwitchOut();

}

Next time we’ll go over the remaining APIs on IHostTask that we haven’t covered yet.

(按:细)

5.
Last time we successfully created a task and started it running. That’s an accomplishment, but there are a couple of details we should get out of the way before we start to dive deeper into the fiber mode implementation. Those are all on IHostTask and are Alert and Join. Once again everything else is trivial or non-important for CoopFiber’s implementation.

Last time we saw a peek at Alert. This was the m_hCurThread member variable that we set when a fiber was switched in. Our Alert implementation uses this:

if(FlagCheck(TASK_FLAG_RUNNING))

{

QueueUserAPC(&MyAPCProc, m_hCurThread, NULL);

}

FlagSet(TASK_FLAG_ALERTED,true);

return(S_OK);

Here we check if the current task is running, and if so we queue an APC to that task. The queued APC is just a simple do-nothing function to wake the task out of its blocking operation. Whether the task is running or not we set the alert bit.

One interesting aspect of this simple sample is that if the user doesn’t schedule the task the alert can never respond. A more complex host may choose to give priority to alerted tasks or schedule them immediately.

The next API to look at is the Join implementation:

HRESULT __stdcall CHostTask::Join(DWORD dwMilliseconds, DWORD option)

{

DWORD result;

if(option & WAIT_ALERTABLE)

{

result = WaitForSingleObjectEx(m_hTaskExitedEvent, dwMilliseconds, true);

}

else

{

result = WaitForSingleObject(m_hTaskExitedEvent, dwMilliseconds);

}

switch(result)

{

case WAIT_OBJECT_0:

return(S_OK);

case WAIT_TIMEOUT:

return(HOST_E_TIMEOUT);

case WAIT_IO_COMPLETION:

return(HOST_E_INTERRUPTED);

}

_ASSERTE(!"Shouldn't reach here");

return(E_FAIL);

}

Here we simply block the current task on an event that is set in our internal CHostTask::ExitTask API. We’ll do an alertable wait if requested, to which the APC queued in Alert will wake us up. Finally Join translates the result of WaitForSingleObject* into the appropriate host HRESULT.

We’ve mentioned ExitTask twice now (once in SwitchIn and here again) so now seems like an appropriate time to cover it. This internal API merely calls the ICLRTask::ExitTask callback, notifying the runtime the task has exited, and updates our internal bookkeeping:

if(m_pCallback!=NULL)

{

m_pCallback->ExitTask();

}

SetEvent(m_hTaskExitedEvent);

FlagSet(TASK_FLAG_EXITED,true);

FlagSet(TASK_FLAG_RUNNING,false);

The one interesting call worth noting here is that we set the event that allows our Joined tasks to wake up. A more complicated host would use Join as an opportunity to deschedule a fiber and would need a more sophisticated mechanism of handling exited tasks.

Well that wraps it up for this edition… There’s only one more piece of unmanaged code before we start getting into the managed world. Next time we’ll delve into the synchronization primitives.

6.
The synchronization primitives are handled off to the runtime by the IHostSyncManager interface. We’ve already provided this to the runtime through our GetHostManager callback on IHostControl. Our IHostSyncManager is implemented on our IHostTaskManager and for the most part we just create objects and hand them off. For example CreateCrst looks like:

*ppCrst = (IHostCrst *)new CHostCriticalSection(this);

if(NULL == *ppCrst)

{

return(E_OUTOFMEMORY);

}

(*ppCrst)->AddRef();

And none of them are very different. Our SetCLRSyncManager does nothing. A more sophisticated host could use the ICLRSyncManager to help perform deadlock detection.

The CoopFiber implements 4 synchronization primitivies: Auto Events, Manual Events, Critical Sections, and a Semaphore. All of these with the exception of the critical section are just thin wrappers over the equivalent OS API. For example looking at CHostManualEvent::Wait:

DWORD result;

if(option & WAIT_ALERTABLE)

{

result = WaitForSingleObjectEx(m_hEvent, dwMilliseconds, true);

}

else

{

result = WaitForSingleObject(m_hEvent, dwMilliseconds);

}

switch(result)

{

case WAIT_OBJECT_0:

return(S_OK);

case WAIT_ABANDONED:

return(HOST_E_ABANDONED);

case WAIT_IO_COMPLETION:

CHostTask::GetCurrentTask()->FlagSet(TASK_FLAG_ALERTED, false);

return(HOST_E_INTERRUPTED);

case WAIT_TIMEOUT:

return(HOST_E_TIMEOUT);

default:

_ASSERTE(!"Shouldn't reach here");

return(E_FAIL);

}

We can see an implementation that is nearly identical to Join we saw last time. Both the auto event and semaphore implementations are nearly identical. More sophisticated hosts could perform deadlock detection below these events (for reader/writer locks and monitors built on top of the events), or they could choose to schedule other fibers on these threads. For this simple implementation we simply block the current thread.

The critical section is the only synchronization primitive which doesn’t simply wrap the OS API. This is because the critical section is owned by a specific thread (unlike the events and semaphore). Because of this the critical section must be aware of the fibers in addition to threads. This will prevent one fiber from acquiring the critical section, and having another fiber re-acquire on the same thread.

The essence of the critical section acquire lives in TryEnter, which Enter builds on:

if(m_holderTask == NULL)

{

CritSecHolder cs(&m_critSec);

// no one holds the critical section

if(m_holderTask == NULL && WaitForSingleObject(m_hEvent, 0) == WAIT_OBJECT_0)

{

// we've acquired the crit section

m_holderTask = curTask;

m_dwEnterCount = 1;

*pbSucceeded = TRUE;

return(S_OK);

}

}

else if(m_holderTask == curTask)

{

CritSecHolder cs(&m_critSec);

// we already hold the critical section

if(m_holderTask == curTask)

{

m_dwEnterCount++;

*pbSucceeded = TRUE;

return(S_OK);

}

}

return(S_OK);

There’s a couple of points to note. First, we use a critical section to protect our state. We don’t want to worry about other threads entering. Also we use an event to block the task if the critical section cannot be acquired. A more sophisticated fiber scheduler would want to re-schedule another fiber rather than block the current thread.

Those are the core synchronization primitives that we’re using in CoopFiber. You can see by limiting the scope of what CoopFiber does we were able to merely rely upon the OS APIs in most circumstances. Next time I’ll start discussing the managed / unmanaged fiber API interface.

7.
At this point we’ve covered nearly all the major components of the unmanaged host. The one remaining detail is the interface that’s used between the managed fiber mode implementation and the unmanaged host. This is all implemented in callbacks.cpp. We essentially expose an API for all the major operations we allow the fiber API to do. These include creating fibers, switching fibers in, aborting fibers, and exiting the current fiber.

The creation of a fiber differs from the CreateTask API we covered before in that we aren’t creating a new thread for this fiber. This fiber is newly created and won’t run until a user switches it in. Aborting fibers is exactly like aborting a thread, but a fiber will need to be switched in to be properly aborted.

Switching in fibers is interesting. Here you’ll find no “SwitchOut” API like the internal unmanaged SwitchOut method on CHostTask. This is because we cannot run managed code on a “switched out” task. It has to have some task to run on! Instead CoopFiber exposes one API for switching in a new task that also switches out the current task.

All of these methods are merely thin wrappers over the functionality we’ve already covered. Two interesting examples of the several functions include InternalCreateFiber and InternalSwitchIn.

First let’s look at InternalCreateFiber:

__declspec(dllexport) PVOID InternalCreateFiber(PVOID startAddr)

{

CHostTask *tmp = CHostTask::CreateTask();

if (tmp == NULL)

{

_ASSERTE(FALSE && "Out of memory in CreateTask!");

return NULL;

}

tmp->SetManagedStart(reinterpret_cast(startAddr));

tmp->AddRef();

tmp->SetFiberAddress(::CreateFiber(0,(LPFIBER_START_ROUTINE )CHostFiberProc,(PVOID)tmp));

return(tmp);

}

This function creates a new CHostTask, sets the start address (which is a delegate passed from managed code), and create a new fiber for the task. We return the task back to managed code which now holds the only reference to it.

The interesting thing to note is the management of the lifetime of the CHostTask. We have 1 reference to it from the managed code. As we’ll see later the managed code uses a new Whidbey feature called SafeHandle. This ensures that when the managed Fiber object is collected our InternalReleaseFiber method will be called.

Next let’s look at InternalSwitchIn:

__declspec(dllexport) void InternalSwitchIn(CHostTask *task)

{

CHostTask *curTask = CHostTask::GetCurrentTask();

_ASSERTE(curTask && task);

curTask->SwitchOut();

task->SwitchIn();

}

Here we get the current task, switch it out, and switch in the task passed in. Again, we have another simple thin wrapper over the APIs we’ve already seen. All of the other callbacks are thin wrapper as well.

So we’ve now covered nearly all the main principals of the unmanaged code. Soon we can start to discuss what the host is going to expose to the managed code author!

8.
It seems like a good time to take a breather and look at what we have going on so far.

First, we’ve started the runtime. We’ve handed our IHostControl off to the runtime. The runtime has called back to our IHostControl and gotten a couple of managers. Those are our task manager and our synchronization manager. Whenever the runtime needs to create a thread it’s called us back, and we’ve created a brand new thread. Sure, we’ve converted it into fibers, but we’ve done no real fiber switching.

Whenever the runtime needed a manual or auto event we’ve given it to it via the synchronization APIs. When it needed to block on it, it called us, and we blocked on it via IHostAutoEvent or IHostManualEvent. If another thread alerted the thread blocked on the event, the runtime called us via the IHostTask APIs, and we queued and APC to it.

If the runtime needed to enter a critical section, it called us, and we blocked or allowed it to enter via the IHostCrst APIs.

We’ve set up everything necessary to replace the CLR’s threading model, but for the most part we’ve merely delegated to the OS APIs. Sure, there were minor differences here and there… But for every task the CLR asks for we create 1 physical thread. And sure every one of those physical threads has already been converted to a fiber, but we never switch those fibers out.

Starting with the next article that’ll start to change.

(按:clean)

9.
The managed Fiber class exists in its own directory in the SDK sample appropriately called Fiber. The Fiber class is designed to be vaguely similar to the managed Thread class. For example, like the Thread has ThreadState the fiber has FiberStates.

The managed fiber class exposes a few significant APIs:

· public static Fiber CreateFiber(FiberStart fs)

· public static Fiber CurrentFiber

· public void SwitchTo()

· public void Abort()

· public void RudeAbort()

· public void Exit()

· public FiberStates FiberState

The CreateFiber API should be obvious. It takes a FiberStart delegate and passes it into the unmanaged InternalCreateFiber API we previously looked at.

CurrentFiber is nearly just as obvious: We store the current fiber in managed thread local storage (or in our case it’s really fiber local storage) so we can easily fetch it. If we haven’t created a managed Fiber object for this fiber yet we’ll get it from the host via a P/Invoke into the InternalGetCurrentFiber API. This simply gets the currently executing CHostTask, AddRef’s it, and returns it. The managed half simply sticks this into our SafeHandle to ensure it’s properly cleaned up.

The last trivial method here is FiberState. This just returns m_FiberStates which the managed Fiber class maintains as the fibers switch through various states.

The interesting APIs here though are actually SwitchTo, Abort, and RudeAbort. We’ll start at looking at SwitchTo.

The first thing we watch out for in the managed implementation is that we don’t try and switch a fiber in on 2 threads:

Fiber curFiber = CurrentFiber;

lock (m_syncObj)

{

if ((m_FiberStates & (FiberStates.Running|FiberStates.Switching)) != 0)

{

throw new FiberStateException("Attempt to switch to a fiber that is running or already switching!");

}

m_FiberStates &= (~FiberStates.Unstarted);

m_FiberStates |= FiberStates.Running;

}

If the fiber was already running, or is currently involved in a switch, then we’ll throw an exception that you cannot currently switch. We’ll then update the fiber we’re trying to switch in so it’s now officially running (this will prevent anyone else from switching it in).

Next we need to make sure that no one races in and tries switching in the fiber we’re switching away from.

lock (curFiber.m_syncObj)

{

m_FiberStates |= FiberStates.Switching;

}

// we''ve marked us as switching, do one last check to see if someone wanted to abort us.

// Anyone after this will get us aborted on our next switch in.

CheckForAbort();

So we update its state to mark that it’s currently switching out. You’ll also notice we’re checking to see if we need to abort a thread. We’ll cover this more in-depth in the next article where I discuss the Abort & RudeAbort implementation.

m_prevFiber = curFiber;

// switch in the fiber the user requested

InternalSwitchIn(m_fiberAddr);

curFiber.OnFiberSwitchIn();

Next we mark what the previous fiber was (we’ll need to update it to remove the Running & Switching bits), and then call back into the unmanaged host to perform the actual switch. This goes into the API that we looked at in Article 7 – we simply switch out the current task, and switch in the new one that we passed in.

Finally we call OnFiberSwitchIn. Again this is another place where we have both a top-half before the switch, and a bottom-half after we’ve switched away and switched back. When we do the switch in if it’s the 1st time a task has been scheduled we’ll end up back at RealFiberStart. If this task has already switched out once, we’ll end up at the curFiber.OnFiberSwitchIn line, but we’ll be on a different fiber (we’ll be on curFiber). In either case we’ll set the state on the previous fiber so that it’ll be schedulable again, and we’ll check the newly switched in fiber to see if it should be aborted.

And that’s how we switch fibers. You first create a fiber, and then you call SwitchTo on the newly created fiber. We largely just do some book keeping, and then call directly into the host to do the switch. It’s that simple.

That takes us through nearly all the basic mechanisms of the fiber mode host. In the next article I’m going to discuss how the Fiber class exposes Abort and RudeAbort APIs. A typical host wouldn’t expose these to managed code, but the CoopFiber sample has it’s reasons for doing so…

10.
In this article we’re going to look at the implementation of Abort and RudeAbort. In the last article I mentioned that a typical host wouldn’t need to expose these to managed code. Why is that? If you have a fiber mode scheduler typically Thread.Abort will be sufficient. Your scheduler will eventually schedule the task, and the task will then be aborted.

But in the CoopFiber sample we don’t have a scheduler. Therefore a user could call Thread.Abort on a switched out fiber, and it would never get scheduled to be aborted. What’s worse is that the call into ICLRTask::Abort will block until the thread gets scheduled again.

To handle this peculiarity when an Abort request comes in the Fiber class first determines whether the task is running or not. If it is already running then we’ll go ahead and abort it right away. Otherwise we’ll set a bit to notify that on the next switch we should abort the task. Here’s the Fiber.Abort implementation:

bool fAbortNeeded = false;

lock (m_syncObj)

{

m_FiberStates |= FiberStates.AbortRequested;

if ((m_FiberStates & FiberStates.Running) != 0 && (m_FiberStates & FiberStates.Switching) == 0)

{

//the current fiber is aborted, so we can initiate the abort now.

fAbortNeeded = true;

}

}

if (fAbortNeeded)

{

InternalAbort(m_fiberAddr);

}

InternalAbort will just turn around and call ICLRTask::Abort on our already switched in task. In our last article we saw our calls to CheckForAbort during task switch in. That code simply looks like:

// Next see if the user requested an abort on this thread.

if ((m_FiberStates & FiberStates.AbortRequested) != 0)

{

Abort();

}

else if ((m_FiberStates & FiberStates.RudeAbortRequested) != 0)

{

RudeAbort();

}

We simply call back to Abort or RudeAbort, which will then call into InternalAbort to finally Abort/RudeAbort the task. The task is now switched in and running so we’ll call into the host to perform the actual abort (which now won’t block because the task is running).

At this point you may be asking yourself “What is this Rude Abort thing?” Rude aborts are actually a new Whidbey feature, and can be accessed one of several ways. The only difference between a rude abort and a normal thread abort is that the rude abort will not execute your catch/finally blocks. In addition to being exposed directly through the hosting APIs rude abort’s can also occur due to escalation policy (also configured via the hosting APIs). That’ll have to be saved for another blog entry…

CoopFiber’s Rude Abort implementation is essentially identical to the Abort implementation. The only difference is that we’ve replaced the work Abort with RudeAbort everywhere.

So that’s it for the Abort/Rude Abort implementation. We’ve almost reached the end of this serious, but there’s one last API that’s worth covering on the managed size: Fiber.Exit. I’ll go over that implementation next article.

11.
Last week we went over the Abort and RudeAbort APIs. This week we’ll go over Fiber.Exit. This API provides a way for you to terminate a running fiber, and demonstrates a use of the unmanaged ICLRTask::ExitTask API. The ExitTask API can only be called on a fiber that is currently switched in which gives us a similar problem to what we had in Abort/RudeAbort. But because the ExitTask call will not block for a long period of time we handle it a little differently than Abort and RudeAbort.

After checking & updating our state like we did in Abort and RudeAbort, the ExitTask API does:

if (this == Fiber.CurrentFiber)

{

// exiting a running task is easy

InternalExitTaskAndThread(fiberAddr);

}

else

{

// only tasks that are running can call ICLRTask::ExitTask.

// We need to switch the task in, call ExitTask, and then

// switch back to our current fiber.

InternalExitTaskAndSwitch(fiberAddr, Fiber.CurrentFiber.m_fiberAddr);

}

Here you can see we take one of two actions. Either we call InternalExitTaskAndTherad() which will cause both the current thread and fiber to exit, or we call into InternalExitTaskAndSwitch.

The first call is pretty self-explanatory. We’re the current thread, we want to let the whole thing die off – this is exactly the same to calling DeleteFiber on the currently running thread.

The second call is a little different. Here we have a fiber that is currently switched out. We want to re-schedule the fiber we’re asking to exit, call ExitTask on it, and then switch back to our previously running fiber. Here we go back to the code in SwitchIn that we looked at in article 4:

if(curTask->FlagCheck(TASK_FLAG_EXITING))

{

_ASSERTE(curTask->GetSwitchingTo());

curTask->ExitTask();

curTask->GetSwitchingTo()->SwitchIn();

}

The task has been requested to exit, we call ICLRTask::Exit on it, and then we switch back to the previously running task having completed the work the user requested.

One note worthy point here is that like the DeleteFiber APIs, the ExitTask API is potentially unsafe. This is because we’re exiting the thread with code higher us still on the stack. This code may have resources to free or other state to clean up. Typically a host would only call ExitTask after all managed code has left the stack.

That brings us to the end of the CoopFiber series. We’ve gone over the smallest set of APIs that are required for implementing fiber mode. We’ve then exposed the APIs to the managed code author to allow them to be in control of the scheduling. To implement a scheduler on top of this would require updating the synchronization primitives so they’d select new tasks and switch them in instead of blocking the thread. And hopefully you’ve walked away from all of this with a better understanding of the interactions between the CLR and the host when running in fiber mode.

This page is powered by Blogger. Isn't yours?