You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 16:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Thu Jun 25, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1295929380,9sleep00-21swapper/007:05:380
1298728963,9sleep30-21swapper/307:06:003
1314528667,8sleep20-21swapper/207:08:042
1296028566,15sleep10-21swapper/107:05:381
1337699270,27cyclictest0-21swapper/212:33:582
1337699270,25cyclictest0-21swapper/211:56:342
1337699260,0cyclictest0-21swapper/207:30:132
1337699230,23cyclictest0-21swapper/211:33:312
1337699230,22cyclictest0-21swapper/211:42:522
1337699220,22cyclictest0-21swapper/209:16:482
13370992216,1cyclictest25-21ksoftirqd/110:34:411
13382992120,1cyclictest0-21swapper/312:30:063
1337699210,20cyclictest0-21swapper/210:27:312
13370992116,1cyclictest25-21ksoftirqd/112:37:591
1336999210,1cyclictest0-21swapper/011:35:280
242862200,2sleep024284-21ssh09:20:160
1337699201,1cyclictest141rcu_preempt09:23:532
1337699200,20cyclictest0-21swapper/210:53:132
1337699200,15cyclictest0-21swapper/212:05:412
13370992017,2cyclictest25-21ksoftirqd/110:55:191
13370992016,4cyclictest38150irq/131-snd_hda10:42:311
1337699190,19cyclictest0-21swapper/211:01:042
1337699190,18cyclictest0-21swapper/210:40:502
1337699190,18cyclictest0-21swapper/210:30:312
1337099192,1cyclictest141rcu_preempt10:16:421
1336999190,2cyclictest0-21swapper/010:03:410
1337699181,0cyclictest141rcu_preempt12:35:332
1337699180,2cyclictest855-21kworker/u8:111:22:012
1337699180,17cyclictest0-21swapper/209:51:002
13370991815,2cyclictest25-21ksoftirqd/109:50:181
13370991813,4cyclictest0-21swapper/112:25:251
1337099180,17cyclictest0-21swapper/109:26:161
1336999180,18cyclictest0-21swapper/010:28:570
211662170,0chrt0-21swapper/111:14:541
1338299174,1cyclictest141rcu_preempt11:02:303
1338299171,4cyclictest8306-21kworker/u8:110:38:053
1338299171,4cyclictest8306-21kworker/u8:110:38:043
1338299171,1cyclictest0-21swapper/308:55:133
13376991717,0cyclictest0-21swapper/210:04:542
13376991713,1cyclictest32-21ksoftirqd/210:07:332
1337699170,2cyclictest0-21swapper/210:46:202
13370991714,2cyclictest25-21ksoftirqd/112:05:261
13370991714,1cyclictest25-21ksoftirqd/111:38:341
1337099171,1cyclictest6128-21diskmemload09:57:451
1337099170,17cyclictest0-21swapper/111:05:491
1337099170,17cyclictest0-21swapper/110:09:091
1336999170,1cyclictest0-21swapper/012:23:560
1336999170,1cyclictest0-21swapper/012:23:560
1336999170,1cyclictest0-21swapper/010:54:150
1337699162,1cyclictest141rcu_preempt11:26:292
1337699160,15cyclictest0-21swapper/211:19:162
1337699160,0cyclictest141rcu_preempt12:01:142
1337099165,9cyclictest22890-21perf07:30:011
1337099164,11cyclictest6128-21diskmemload10:35:351
1337099164,11cyclictest6128-21diskmemload10:35:351
1337099162,13cyclictest6128-21diskmemload10:25:431
1337099161,6cyclictest22549-21perf11:34:591
13370991614,2cyclictest25-21ksoftirqd/112:32:111
13370991614,1cyclictest0-21swapper/110:10:471
1337099160,14cyclictest6128-21diskmemload11:15:521
1337099160,11cyclictest20128-21vmstat07:20:261
1336999164,6cyclictest4535-21perf11:05:000
1336999160,1cyclictest0-21swapper/009:37:280
1336999160,1cyclictest0-21swapper/009:29:440
1338299156,7cyclictest23353-21/usr/sbin/munin09:20:153
1337699150,14cyclictest0-21swapper/210:56:322
1337699150,13cyclictest0-21swapper/209:26:232
1337099154,9cyclictest4583-21sh11:43:131
1337099151,9cyclictest25-21ksoftirqd/108:00:201
13370991513,1cyclictest0-21swapper/111:00:141
1337099150,4cyclictest0-21swapper/109:30:241
1337099150,15cyclictest0-21swapper/112:18:541
1337099150,14cyclictest0-21swapper/112:21:501
1337099150,14cyclictest0-21swapper/112:21:501
1336999151,11cyclictest0-21swapper/011:25:130
13382991411,2cyclictest15133-21expr11:30:153
1337699149,4cyclictest19279-21taskset12:10:472
13376991410,3cyclictest291irq_work/209:42:292
1337699140,14cyclictest0-21swapper/212:28:192
1337699140,13cyclictest0-21swapper/212:15:152
1337099142,2cyclictest141rcu_preempt11:23:221
13370991411,2cyclictest11658-21munin-run10:50:001
1337099140,4cyclictest6128-21diskmemload11:56:271
13369991412,1cyclictest12113-21taskset09:13:070
13369991411,2cyclictest18992-21grep07:20:150
1336999140,7cyclictest0-21swapper/008:30:120
1336999140,13cyclictest8209-21cat12:05:000
1336999140,12cyclictest0-21swapper/010:48:020
1338299132,8cyclictest0-21swapper/309:55:163
13382991310,2cyclictest9823-21cat12:05:183
13382991310,2cyclictest9351-21sed11:45:243
13382991310,2cyclictest8767-21ssh11:06:473
13382991310,2cyclictest20388-21grep08:30:153
13382991310,2cyclictest16439-21head09:15:213
13382991310,2cyclictest14553-21awk09:15:003
1338299130,3cyclictest0-21swapper/309:35:123
1338299130,12cyclictest10326-21grep10:10:143
1337699139,2cyclictest25980-21wc08:40:252
13376991311,1cyclictest28076-21pmu-power10:20:222
13376991310,2cyclictest934-21grep12:20:152
13376991310,2cyclictest934-21grep12:20:142
13376991310,2cyclictest16689-21grep08:20:252
1337699130,1cyclictest0-21swapper/210:10:492
1337699130,12cyclictest0-21swapper/209:14:292
1337099139,4cyclictest25-21ksoftirqd/109:38:261
1337099139,1cyclictest4449-21grep07:55:251
1337099138,4cyclictest2267-21grep09:45:261
1337099137,3cyclictest141rcu_preempt09:18:041
13370991310,2cyclictest8599-21ssh09:10:261
13370991310,2cyclictest8192-21chmod12:05:001
13370991310,2cyclictest24062-21grep07:30:161
13370991310,2cyclictest14108-21sed07:10:161
1337099130,1cyclictest6128-21diskmemload10:04:481
1337099130,1cyclictest0-21swapper/109:42:421
1336999139,3cyclictest8530-21head09:30:170
1336999139,2cyclictest666-21cat08:55:250
1336999133,7cyclictest0-21swapper/012:15:240
1336999132,6cyclictest18933-21awk10:35:000
13369991310,2cyclictest19501-21dump-pmu-power08:30:000
13369991310,2cyclictest15722-21idleruntime-cro07:15:000
1336999130,7cyclictest32083-21grep12:00:000
1338299129,2cyclictest9496-21head09:50:163
1338299129,2cyclictest9419-21grep08:05:253
1338299129,2cyclictest4719-21grep09:05:183
1338299129,2cyclictest4719-21grep09:05:173
1338299129,2cyclictest4393-21cat07:55:253
1338299129,2cyclictest32269-21sh09:44:593
1338299129,2cyclictest32246-21grep11:40:163
1338299129,2cyclictest30925-21grep08:50:253
1338299129,2cyclictest24127-21tail07:30:163
1338299129,2cyclictest18783-21grep12:10:253
1338299129,2cyclictest13842-21grep08:15:213
13382991211,1cyclictest39-21ksoftirqd/311:53:253
13382991210,1cyclictest10550irq/122-ahci[0012:30:003
1338299120,11cyclictest11902-21cut10:30:183
1337699129,2cyclictest9164-21wc11:45:242
1337699129,2cyclictest5109-21strings09:05:202
1337699129,2cyclictest5109-21strings09:05:202
1337699129,2cyclictest2890-21wc09:00:222
1337699129,2cyclictest20995-21grep07:25:152
1337699129,2cyclictest18811-21wc08:25:272
1337699129,2cyclictest15282-21tail07:10:272
1337699129,2cyclictest1115-21sort07:50:172
1337699122,2cyclictest141rcu_preempt11:35:522
1337699120,3cyclictest0-21swapper/211:09:212
1337699120,11cyclictest0-21swapper/210:37:192
1337699120,11cyclictest0-21swapper/210:37:182
1337099129,2cyclictest31944-21tr08:55:141
1337099129,2cyclictest28162-21wc08:45:211
1337099129,2cyclictest25-21ksoftirqd/112:15:001
1337099129,2cyclictest14025-21awk11:30:001
1337099129,2cyclictest12753-21grep08:15:151
1337099128,3cyclictest27430-21grep07:35:231
1337099120,1cyclictest141rcu_preempt09:20:401
1337099120,11cyclictest0-21swapper/111:47:051
1336999129,2cyclictest4713-21grep09:05:180
1336999129,2cyclictest4713-21grep09:05:170
1336999129,2cyclictest31386-21ls08:55:010
1336999129,2cyclictest28585-21tail10:40:170
1336999129,2cyclictest27350-21grep10:20:170
1336999129,2cyclictest23747-21grep08:35:250
1336999129,2cyclictest22669-21cat11:35:000
1336999129,2cyclictest20597-21grep10:55:140
1336999129,2cyclictest17869-21grep09:55:180
1336999128,3cyclictest24728-21grep09:40:140
1336999123,7cyclictest20194-21/usr/sbin/munin10:35:150
1336999123,7cyclictest20194-21/usr/sbin/munin10:35:150
1336999123,5cyclictest10650-21grep09:50:250
1336999121,5cyclictest15088-21aten2_r7power_c09:15:130
1336999120,6cyclictest32621-21basename07:49:590
1336999120,11cyclictest18620-21grep10:15:140
1338299119,1cyclictest6934-21grep11:25:183
1338299119,1cyclictest30545-21expr11:20:133
1338299119,1cyclictest23371-21grep08:35:213
1338299118,2cyclictest6862-21ssh12:04:083
1338299118,2cyclictest521-21ssh10:04:433
1338299118,2cyclictest5198-21cat08:00:123
1338299118,2cyclictest5024-21ssh10:06:423
1338299118,2cyclictest4529-21head10:45:173
1338299118,2cyclictest4469-21ssh09:47:043
1338299118,2cyclictest3378-21rm09:27:033
1338299118,2cyclictest30718-21tr07:45:133
1338299118,2cyclictest29772-21sed07:40:263
1338299118,2cyclictest2820-21ls09:00:223
1338299118,2cyclictest27393-21chrt12:36:023
1338299118,2cyclictest26964-21grep07:35:203
1338299118,2cyclictest24973-21sed11:35:253
1338299118,2cyclictest23714-21sed11:15:253
1338299118,2cyclictest20830-21cut07:25:133
1338299118,2cyclictest18500-21tr07:20:133
1338299118,2cyclictest18328-21basename10:15:143
1338299118,2cyclictest11234-21rm09:31:373
1338299118,2cyclictest10623-21cat08:10:143
13382991111,0cyclictest0-21swapper/310:52:083
1338299110,2cyclictest0-21swapper/310:57:283
1338299110,10cyclictest0-21swapper/310:40:013
1337699118,2cyclictest7502-21ssh09:30:062
1337699118,2cyclictest32621-21ssh09:45:062
1337699118,2cyclictest2837-21hostname07:55:132
1337699118,2cyclictest19495-21wc10:15:192
1337699118,2cyclictest17965-21grep09:55:182
1337699118,2cyclictest16374-21grep09:35:132
1337699117,3cyclictest5252-21grep08:00:142
1337699113,6cyclictest12849-21grep08:15:152
1337699113,4cyclictest225-21systemd-journal07:45:002
1337699110,9cyclictest0-21swapper/211:50:292
1337699110,10cyclictest0-21swapper/211:10:012
1337099118,2cyclictest4784-21cut09:05:171
1337099118,2cyclictest4784-21cut09:05:171
1337099118,2cyclictest28097-21cat07:40:121
1337099118,2cyclictest26690-21rm10:20:141
1337099118,2cyclictest25589-21wc08:40:201
1337099118,2cyclictest25-21ksoftirqd/108:50:161
1337099118,2cyclictest17681-21sed07:15:261
1337099118,2cyclictest16111-21sh10:51:591
1337099118,2cyclictest10148-21cut08:10:001
1337099113,2cyclictest141rcu_preempt11:50:181
1336999118,2cyclictest9580-21tail12:05:170
1336999118,2cyclictest7057-21cat11:45:010
1336999118,2cyclictest5192-21cat08:00:110
1336999118,2cyclictest4818-21sh11:05:040
1336999118,2cyclictest363-21taskset07:50:090
1336999118,2cyclictest3302-21sed09:00:250
1336999118,2cyclictest22544-21chrt07:25:530
1336999118,2cyclictest21907-21ssh12:13:000
1336999118,2cyclictest18890-21ssh11:51:150
1336999118,2cyclictest15177-21sh12:29:130
1336999118,2cyclictest1247-21ssh10:05:050
1336999116,4cyclictest16634-21apache_processe12:30:110
1336999112,8cyclictest2179-21timerwakeupswit09:45:260
1336999110,5cyclictest0-21swapper/011:20:120
1338299108,2cyclictest5073-21sh10:25:473
1338299108,2cyclictest27552-21expr10:20:173
1338299108,2cyclictest1089-21uniq07:50:163
1338299108,1cyclictest29213-21cat08:50:003
1338299108,1cyclictest27245-21ssh12:15:373
1338299108,1cyclictest24710-21sh11:55:143
1338299108,1cyclictest2470-21grep12:20:263
1338299108,1cyclictest2470-21grep12:20:253
1338299108,1cyclictest18482-21grep08:25:203
1338299108,1cyclictest16504-21sh11:11:153
1338299108,1cyclictest15228-21cat08:20:133
1338299108,1cyclictest13840-21grep07:10:163
1337699108,1cyclictest7856-21cut08:05:132
1337699108,1cyclictest27395-21sed08:45:142
1337699108,1cyclictest25777-21cut07:35:122
1337699108,1cyclictest23909-21chrt08:35:472
1337699108,1cyclictest18462-21wget07:20:132
1337699108,1cyclictest10877-21cut08:10:162
1337699107,2cyclictest21766-21chrt08:33:462
1337699106,3cyclictest32-21ksoftirqd/207:45:182
1337699106,2cyclictest29516-21tr08:50:152
1337099108,1cyclictest17602-21cat08:25:151
1337099108,1cyclictest11263-21memory08:10:191
1337099107,2cyclictest23886-21chrt08:35:301
1337099107,2cyclictest19801-21cut08:30:121
1337099107,2cyclictest1918-21tr09:00:131
1337099107,2cyclictest1900-21cat07:50:261
1337099107,2cyclictest15122-21cut08:20:131
1336999108,1cyclictest9834-21taskset08:09:090
1336999108,1cyclictest3146-21expr07:55:140
1336999108,1cyclictest28886-21expr07:40:190
1336999108,1cyclictest24785-21basename12:35:140
1336999108,1cyclictest17332-21readlink07:15:220
1336999108,1cyclictest13833-21expr11:10:140
1336999108,1cyclictest13571-21cut08:15:200
1336999108,1cyclictest11175-21grep08:10:190
1336999107,2cyclictest7255-21ssh11:45:060
1336999107,2cyclictest25834-21processes08:40:220
1336999107,2cyclictest23983-21uniq07:30:150
1336999107,2cyclictest15247-21tr08:20:130
1336999107,2cyclictest11293-21ntpq10:10:200
1336999100,9cyclictest24784-21sshd11:16:120
133829998,1cyclictest24536-21apache_volume08:40:123
133829997,1cyclictest15998-21apache_volume07:15:113
133769998,1cyclictest16064-21wget07:15:142
133769997,1cyclictest10112sleep208:56:022
133709996,2cyclictest31227-21tail07:45:161
133699997,1cyclictest25873-21egrep07:35:120
133699996,2cyclictest27666-21head08:45:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional