Archive

Archive for the ‘Tech.Notes’ Category

招商证券股票day数据格式分析

January 8th, 2012 No comments

今天没事想把招商证券保存的股票数据提取出来, 后来发现招商证券的软件是采用是通达信做的, 这样直接通过查找通达信的数据格式就搞定了.

主要参考: http://alantop.5166.info/gpdatasoft/gpformat/gpformat.htm#_Toc224993378

通达信日线

每条数据文件占32个字节。每四个字节代表一个变量。如下所示:

 

struct stockDay
{
uint32_t date; //日期
uint32_t open; //开盘价,单位:分
uint32_t high; //最高价,单位:分
uint32_t low; //最低价,单位:分
uint32_t close; //收盘价,单位:分
float amount; //交易金额,单位:元
uint32_t vol; //成交量,单位:股
int32_t reserv; //保留,有时用来保存上一交易日收盘价
};

我分别使用C和Go写了个简单的提取程序.

C代码:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
struct stockDay
{
uint32_t date; //日期
uint32_t open; //开盘价,单位:分
uint32_t high; //最高价,单位:分
uint32_t low; //最低价,单位:分
uint32_t close; //收盘价,单位:分
float amount; //交易金额,单位:元
uint32_t vol; //成交量,单位:股
int32_t reserv; //保留,有时用来保存上一交易日收盘价
};

void printDay(struct stockDay *day)
{
printf(“日期: %d, “, day->date);
printf(“开盘价: %.2f, 最高价: %.2f, 最低价: %.2f, 收盘价: %.2f, “, day->open/100.0, day->high/100.0, day->low/100.0, day->close/100.0);
printf(“交易金额: %f, 交易量: %d\n”, day->amount, day->vol);
}

int main()
{
FILE *p;
int i = 0;
char *file = “./sh600529.day”;
struct stockDay *day = malloc(sizeof(struct stockDay));

p = fopen(file, “r”);
if (p) {
fseek(p, 0, SEEK_END);
uint64_t iFileLen = ftell(p)/(sizeof(struct stockDay));
rewind(p);
for (i = 0; i < iFileLen; i++) {
memset(day, 0, sizeof(struct stockDay));
fread(day, sizeof(struct stockDay), 1, p);
printDay(day);
}

} else {
printf(“Read File: %s Error!\n”, file);
}

free(day);
fclose(p);
return 0;
}

Go的代码:

package main

import (
“bytes”
“encoding/binary”
“fmt”
“io/ioutil”
“unsafe”
)

type StockDay struct {
date uint32 //日期
open uint32 //开盘价,单位:分
high uint32 //最高价,单位:分
low uint32 //最低价,单位:分
cls uint32 //收盘价,单位:分
amount float32 //交易金额,单位:元
vol uint32 //成交量,单位:股
reserv int32 //保留,有时用来保存上一交易日收盘价
}

func main() {
contents, _ := ioutil.ReadFile(“./sh887001.day”)
//println(string(contents));
var day StockDay
size := int(unsafe.Sizeof(day))
fmt.Printf(“sizeof: %d, days: %d\n”, size, len(contents)/size)
for i := 0; i < len(contents); i += 32 {
buf := bytes.NewBuffer(contents[i : i+4])
err := binary.Read(buf, binary.LittleEndian, &day.date)
if err != nil {
fmt.Println(“binary.Read failed: “, err)
}
buf = bytes.NewBuffer(contents[i+4 : i+8])
err = binary.Read(buf, binary.LittleEndian, &day.open)

buf = bytes.NewBuffer(contents[i+8 : i+12])
err = binary.Read(buf, binary.LittleEndian, &day.high)

buf = bytes.NewBuffer(contents[i+12 : i+16])
err = binary.Read(buf, binary.LittleEndian, &day.low)

buf = bytes.NewBuffer(contents[i+16 : i+20])
err = binary.Read(buf, binary.LittleEndian, &day.cls)

buf = bytes.NewBuffer(contents[i+20 : i+24])
err = binary.Read(buf, binary.LittleEndian, &day.amount)

buf = bytes.NewBuffer(contents[i+24 : i+28])
err = binary.Read(buf, binary.LittleEndian, &day.vol)

fmt.Printf(“%v\n”, day)
}
}

 

代码也贴在Github上: https://gist.github.com/1578661

ps: 目前5分钟线的月份和日期对不上, 不知道哪位高手搞过?

–EOF–

 

golang Byte数组转换为浮点数

January 8th, 2012 No comments

今天没事想用go从文件中提取些二进制数据,并把数据转换为相关的数据类型, 就找到一个把数组转换为float类型的数据。主要用到了encoding/binary包里面的read函数, 它可以根据你数据的大小端得到相关的数据类型。

主要代码如下:

package main

import (
“bytes”
“encoding/binary”
“fmt”
)

func main() {
var pi float64
b := []byte{0×18, 0x2d, 0×44, 0×54, 0xfb, 0×21, 0×09, 0×40}
buf := bytes.NewBuffer(b)
err := binary.Read(buf, binary.LittleEndian, &pi)
if err != nil {
fmt.Println(“binary.Read failed:”, err)
}
fmt.Println(pi)
}

下面是go里面uint32在大小端的分别实现:

bvcom@bvcomtv:~$ godoc -src encoding/binary Uint32
// A ByteOrder specifies how to convert byte sequences into
// 16-, 32-, or 64-bit unsigned integers.
type ByteOrder interface {
Uint32([]byte) uint32
// contains filtered or unexported methods
}

func (littleEndian) Uint32(b []byte) uint32 {
return uint32(b[0]) | uint32(b[1])<<8 | uint32(b[2])<<16 | uint32(b[3])<<24
}

func (bigEndian) Uint32(b []byte) uint32 {
return uint32(b[3]) | uint32(b[2])<<8 | uint32(b[1])<<16 | uint32(b[0])<<24
}

参考:
Convert 8 byte array to int64? Options:  http://groups.google.com/group/golang-nuts/browse_thread/thread/7f541090fd7d1bd4

Reading byte array as another datatype Options: http://groups.google.com/group/golang-nuts/browse_thread/thread/fc00115bf5b35757

http://golang.org/src/pkg/encoding/binary/binary.go?s=3435:3501#L117

–EOF–

buildroot编译出错信息

December 20th, 2011 No comments

1. PATH环境变量有当前目录
错误信息:

border@b0rder:/work/tss/src/lichee/buildroot$ ./build.sh -p sun4i -m buildroot

You seem to have the current working directory in your PATH environment variable. This doesn’t work.

make: *** [dependencies] 错误 1

解决方法,

border@b0rder:/work/tss/src/lichee/buildroot$ echo $PATH .:/home/border/bin/jdk1.6.0_26/bin:.:.:/home/border/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/arm/arm-2010.09/bin:/home/border/go/bin://home/border/bin:/usr/local/arm/arm-2009q3/bin:/home/border/bin:/work/vc1000/src/android-vc1000/out/host/linux-x86/bin:/home/border/bin:/home/border/work/depot_tools:/home/border/work/app/google_appengine:/work/vc1000/src/android-vc1000/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin:/home/border/work/android/eclipse-x64:/home/border/work/ai/python/:/home/border/work/ai/python/tools

上面的.表示当前目录信息, 去掉. 后才能正常编译。

border@b0rder:/work/tss/src/lichee/buildroot$ export PATH=/home/border/bin/jdk1.6.0_26/bin:/home/border/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/arm/arm-2010.09/bin:/home/border/go/bin://home/border/bin:/usr/local/arm/arm-2009q3/bin:/home/border/bin:/work/vc1000/src/android-vc1000/out/host/linux-x86/bin:/home/border/bin:/home/border/work/depot_tools:/home/border/work/app/google_appengine:/work/vc1000/src/android-vc1000/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin:/home/border/work/android/eclipse-x64:/home/border/work/ai/python/:/home/border/work/ai/python/tools border@b0rder:/work/tss/src/lichee/buildroot$ echo $PATH /home/border/bin/jdk1.6.0_26/bin:/home/border/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/arm/arm-2010.09/bin:/home/border/go/bin://home/border/bin:/usr/local/arm/arm-2009q3/bin:/home/border/bin:/work/vc1000/src/android-vc1000/out/host/linux-x86/bin:/home/border/bin:/home/border/work/depot_tools:/home/border/work/app/google_appengine:/work/vc1000/src/android-vc1000/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin:/home/border/work/android/eclipse-x64:/home/border/work/ai/python/:/home/border/work/ai/python/tools

2. makeinfo
错误信息:

border@b0rder:/mnt/ubuntu32/work/tss/src/lichee$ ./build.sh -p sun4i -m buildroot

You must install ‘makeinfo’ on your build machine makeinfo is usually part of the texinfo package in your distribution

make: *** [dependencies] 错误 1

解决方法:

sudo apt-get install texinfo

–EOF–

Kernel debug with kgtp on android

November 1st, 2011 No comments

KGTP is a realtime and lightweight Linux Kernel GDB debugger and tracer. It makes Linux Kernel supply a GDB remote debug interface. Then GDB in current machine or remote machine can debug and trace Linux through GDB tracepoint without stopping the Linux Kernel. And even if the board doesn’t have GDB on it and doesn’t have interface for remote debug. It can debug the Linux Kernel using offline debug. Now, it supports X86-32, X86-64, MIPS and ARM.

This article describes how to use kgtp debug linux kernel on android.

Kernel Building

 General setup  --->
     [ * ] Prompt for development and/or incomplete code/drivers
     [ * ] Kprobe
Kernel hacking  --->
     [ * ] Compile the kernel with debug info
     [ * ] Compile the kernel with frame pointers

Building KGTP

Config KGTP Makefile

KERNELDIR := /work/vc1000/src/kernel-vc1000-2.3
ARCH=arm
CROSS_COMPILE=/usr/local/arm/arm-2009q3/bin/arm-none-linux-gnueabi-

error: ‘GTP_VAR_RDTSC_ID’ undeclared(latest version fixed By: teawater )

  CC [M]  /home/border/work/kernel/kgtp/trunk/gtp.o
/home/border/work/kernel/kgtp/trunk/gtp.c: In function 'gtp_gdbrsp_qtv':
/home/border/work/kernel/kgtp/trunk/gtp.c:6318: error: 'GTP_VAR_RDTSC_ID' undeclared (first use in this function)
/home/border/work/kernel/kgtp/trunk/gtp.c:6318: error: (Each undeclared identifier is reported only once
/home/border/work/kernel/kgtp/trunk/gtp.c:6318: error: for each function it appears in.)
/home/border/work/kernel/kgtp/trunk/gtp.c:6320: error: implicit declaration of function 'rdtscll'

* solution:

diff --git a/trunk/gtp.c b/trunk/gtp.c
index 4d0c9a2..75c145e 100644
--- a/trunk/gtp.c
+++ b/trunk/gtp.c
@@ -6315,11 +6315,13 @@ gtp_gdbrsp_qtv(char *pkg)
                if (num == GTP_VAR_CLOCK_ID) {
                        val = (uint64_t)GTP_LOCAL_CLOCK;
                        goto output_value;
+#ifdef CONFIG_X86
                } else if (num == GTP_VAR_RDTSC_ID) {
                        unsigned long long a;
                        rdtscll(a);
                        val = (uint64_t)a;
                        goto output_value;
+#endif
                } else if (num == GTP_VAR_XTIME_SEC_ID
                           || num == GTP_VAR_XTIME_NSEC_ID) {
                        struct timespec time

Complie and Install KGTP

make
sudo su
adb push gtp.ko /system/vendor/lib
exit

KGTP Running On Android

#Open the KGTP interface in current machine.
su
cd /system/vendor/lib
insmod gtp.ko
lsmod
nc -l -p 1234 < /sys/kernel/debug/gtp > /sys/kernel/debug/gtp

Host PC

cd /work/vc1000/src/kernel-vc1000-2.3
make -j8

Network connect to gtp

# symbian use "set gnutarget elf32-littlearm-symbian"
# vxworks use "set gnutarget elf32-littlearm-vxworks"
gdb-release -ex "set gnutarget elf32-littlearm" -ex "file ./vmlinux"

# if you want see the debug info
(gdb) set debug remote 1

# connection your remote device
(gdb) target remote 192.168.2.213:1234

USB connect to gtp

sudo su
# forward socket(adb forward <local> <remote>)
adb forward tcp:1234 tcp:1234
exit
gdb-release -ex "set gnutarget elf32-littlearm" -ex "file ./vmlinux"
# connection your remote device
(gdb) target remote 127.0.0.1:1234

Debugging with gtp

(gdb) trace vfs_readdir
Tracepoint 1 at 0xc02289f0: file /build/buildd/linux-2.6.35/fs/readdir.c, line 23.
(gdb) actions
Enter actions for tracepoint 1, one per line.
End with a line saying just "end".
>collect $reg
>end
(gdb) tstart

Change To Android Device Shell And Run ls Command * Android Device

$ ls

Back To Host PC GDB Shell * Host PC

(gdb) shell ls
vmlinux-2.6.35-30-generic
(gdb) tstop
(gdb) tfind
Found trace frame 0, tracepoint 1
#0  vfs_readdir (file=0x0, filler=0x163d8ae3, buf=0x18c0) at /build/buildd/linux-2.6.35/fs/readdir.c:23
23      {

Good luck, Happy Hacking…

 

Very grateful for teawater

References

kgtp Linux Kernel GDB Tracepoint module

kgtp Quick start

kgtp How to

How To Use KGTP In Android

GDB Tracepoints

移植有线网络到Android2.3

August 5th, 2011 No comments

  近来做Android有线网络移植,主要参考了Nicu PavelEthernet patch for Android Gingerbread 按照Nicu Pavel 的方法在我的板子上死活运行不起来, 在“设置”里面点击“有线网络配置”,通过logcat 查看一直显示我的”ethernet”服务没有起来(I/EthernetManager( 648): Init Ethernet Manager, service: null)。

又仔细检查了一下patch,确定打的补丁没有问题,只能在深入的分析logcat日志。

在日志中发现有几处可能和ethernet相关的地方。

  • I/ethernet( 2060): Loading ethernet jni class
  • I/SystemServer( 2136): NetStat Service
  • I/SystemServer( 2136): NetworkManagement Service
  • I/SystemServer( 2136): Connectivity Service

分别查看了这几个文件和相关的服务, 排除了前三个的可能,就拿“Connectivity Service”开始, 打开frameworks/base/services/java/com/android/server/ConnectivityService.java文件:

  • 打开dbg信息: private static final boolean DBG = true;
  • 在ConnectivityService构造函数里面加了几句打印信息
    String[] naStrings = context.getResources().getStringArray(
            com.android.internal.R.array.networkAttributes);
    for (String naString : naStrings) {
        try {
            NetworkAttributes n = new NetworkAttributes(naString);
            // Add Debug Info
            Slog.v(TAG, "[ConnectivityService.java] NetworkAttributes naString: " + naString  + "  type: " + n.mType);
            if (n.mType > ConnectivityManager.MAX_NETWORK_TYPE) {
                Slog.e(TAG, "Error in networkAttributes - ignoring attempt to define type " +
                        n.mType);
                continue;
            }
            if (mNetAttributes[n.mType] != null) {
                Slog.e(TAG, "Error in networkAttributes - ignoring attempt to redefine type " +
                        n.mType);
                continue;
            }
            if ((n.mType != ConnectivityManager.TYPE_ETHERNET) && (mRadioAttributes[n.mRadio] == null)) {
                Slog.e(TAG, "Error in networkAttributes - ignoring attempt to use undefined " +
                        "radio " + n.mRadio + " in network type " + n.mType);
                continue;
            }
            mNetAttributes[n.mType] = n;
            mNetworksDefined++;
        } catch(Exception e) {
            Slog.e(TAG, "wrong dev exception " + e);
            // ignore it - leave the entry null
        }
    }

得到相关的打印信息如下:

I/SystemServer( 2136): NetStat Service
I/SystemServer( 2136): NetworkManagement Service
I/SystemServer( 2136): Connectivity Service
V/ConnectivityService( 2136): ConnectivityService starting up
V/ConnectivityService( 2136): [ConnectivityService.java] NetworkAttributes naString: wifi,1,1,1  type: 1
V/ConnectivityService( 2136): [ConnectivityService.java] NetworkAttributes naString: mobile,0,0,0  type: 0
V/ConnectivityService( 2136): [ConnectivityService.java] NetworkAttributes naString: mobile_mms,2,0,2  type: 2
V/ConnectivityService( 2136): [ConnectivityService.java] NetworkAttributes naString: mobile_supl,3,0,2  type: 3
V/ConnectivityService( 2136): [ConnectivityService.java] NetworkAttributes naString: mobile_hipri,5,0,3  type: 5

除了ethernet其他的信息都有,这个比较奇怪,但是开头的那句, 就是去掉网络的配置信息,应该没有什么问题。

String[] naStrings = context.getResources().getStringArray(
        com.android.internal.R.array.networkAttributes);

又去仔细检测了一遍frameworks/base/core/res/res/values/config.xml文件中的配置信息, 没有问题。

<string-array translatable="false" name="networkAttributes">
<item>"wifi,1,1,1"</item>
<item>"mobile,0,0,0"</item>
<item>"mobile_mms,2,0,2"</item>
<item>"mobile_supl,3,0,2"</item>
<item>"mobile_hipri,5,0,3"</item>
<item>"ethernet,9,9,1"</item>
</string-array>

看到这里就比较比较奇怪了,配置文件中明明有ethernet但是,通过context去得的数据就偏偏没有ethernet, 再去ConnectivityManager.java中检查了一下 TYPE_ETHERNET的值,的确和配置文件中的数值一样,MAX_RADIO_TYPE和MAX_NETWORK_TYPE也都是TYPE_ETHERNET.

郁闷了半天也想不出来咋搞,习惯性的永grep -Rw 搜索了一下networkAttributes。 在搜索的结果中发现了”device/samsung/smdkv210/overlay/frameworks/base/core/res/res/values/config.xml”这个地址。

打开文件,发现这个配置里面的networkAttributes没有相关ethernet信息,可能是把framework里面的配置信息覆盖掉了.

在config.xml中增加ethernet相关的信息,编译,运行。OK,果真是这个问题引起的, 唉。