You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 15:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sat Mar 14, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21955210165,31sleep00-21swapper/007:07:380
2193529342,46sleep10-21swapper/107:07:231
2183828859,9sleep30-21swapper/307:06:093
2202428068,8sleep20-21swapper/207:08:332
22222992522,2cyclictest25-21ksoftirqd/111:28:081
2222099250,5cyclictest0-21swapper/012:04:360
2222999240,1cyclictest0-21swapper/312:29:123
2222099242,21cyclictest0-21swapper/009:41:590
2222099240,0cyclictest0-21swapper/009:29:020
2222299233,17cyclictest0-21swapper/111:55:121
22222992219,1cyclictest0-21swapper/112:14:121
22220992219,2cyclictest22430-21sensors10:50:240
2222099220,2cyclictest0-21swapper/010:38:510
2222999212,19cyclictest0-21swapper/310:07:003
2222699210,20cyclictest0-21swapper/212:14:552
26862200,0chrt0-21swapper/109:20:581
22229992013,5cyclictest22653-21sh11:30:083
2222699201,18cyclictest0-21swapper/210:46:592
2222699201,18cyclictest0-21swapper/210:46:592
2222099202,18cyclictest0-21swapper/009:52:160
31092190,2sleep10-21swapper/110:58:531
2222699191,3cyclictest0-21swapper/207:25:132
2222699190,1cyclictest141rcu_preempt11:53:012
2222699190,19cyclictest0-21swapper/210:34:122
2222699190,10cyclictest15227-21diskmemload09:20:132
2222299191,13cyclictest15227-21diskmemload11:40:151
2222299190,2cyclictest0-21swapper/111:46:491
2222099192,17cyclictest0-21swapper/010:42:230
2222999180,1cyclictest0-21swapper/309:33:173
2222699180,18cyclictest0-21swapper/210:37:512
2222699180,17cyclictest0-21swapper/210:23:072
2222699180,11cyclictest0-21swapper/207:20:132
22222991815,2cyclictest25-21ksoftirqd/112:25:271
22220991817,1cyclictest0-21swapper/010:04:160
2222999170,17cyclictest0-21swapper/311:07:373
2222699170,17cyclictest0-21swapper/210:04:302
22222991713,3cyclictest3650-21aten2_r7power_e08:45:131
2222299170,1cyclictest141rcu_preempt10:49:281
2222299170,1cyclictest141rcu_preempt10:49:281
2222099174,11cyclictest17359-21ssh12:25:130
2222099173,5cyclictest0-21swapper/011:55:120
2222099170,5cyclictest0-21swapper/010:07:130
22229991616,0cyclictest0-21swapper/312:34:263
22229991616,0cyclictest0-21swapper/309:29:493
22229991615,1cyclictest0-21swapper/311:45:153
22229991613,2cyclictest39-21ksoftirqd/308:00:203
2222999160,16cyclictest0-21swapper/312:35:413
2222999160,16cyclictest0-21swapper/311:38:553
2222699160,15cyclictest0-21swapper/210:50:242
22222991612,1cyclictest25-21ksoftirqd/110:08:021
2222299160,15cyclictest22532-21df_abs11:10:141
2222099163,7cyclictest15227-21diskmemload09:32:500
22220991613,2cyclictest4100-21idleruntime-cro07:39:590
2222999153,6cyclictest24689-21entropy09:15:153
2222999152,13cyclictest0-21swapper/310:51:313
22229991511,2cyclictest39-21ksoftirqd/309:55:363
22229991511,2cyclictest29964-21ntpq08:30:213
2222999150,2cyclictest0-21swapper/310:49:443
2222999150,2cyclictest0-21swapper/310:49:443
2222699150,3cyclictest0-21swapper/211:36:562
2222299152,8cyclictest15227-21diskmemload11:52:141
22222991511,1cyclictest25-21ksoftirqd/109:54:521
22222991510,1cyclictest25-21ksoftirqd/109:17:231
2222099153,9cyclictest15227-21diskmemload11:51:510
2222099151,4cyclictest0-21swapper/011:25:130
2222999141,3cyclictest141rcu_preempt09:45:183
2222999140,1cyclictest0-21swapper/310:44:253
2222999140,13cyclictest0-21swapper/311:22:113
22226991412,2cyclictest32-21ksoftirqd/210:06:382
2222699140,14cyclictest0-21swapper/212:39:512
2222299149,4cyclictest9143-21tr12:20:141
2222299144,9cyclictest3768-21taskset10:20:161
2222299142,10cyclictest0-21swapper/110:10:131
22222991412,1cyclictest11403-21ssh10:44:361
2222299141,1cyclictest25630-21ssh12:30:131
22222991411,2cyclictest17157-21wc09:10:211
22222991410,3cyclictest8090-21munin-plugin-st08:55:001
22222991410,3cyclictest5492-21ssh12:36:581
22222991410,3cyclictest26061-21tr07:15:221
22222991410,3cyclictest19407-21cut08:10:161
22222991410,2cyclictest711-21grep07:30:181
22222991410,2cyclictest3406-21wc08:45:001
2222299140,1cyclictest0-21swapper/110:39:371
2222299140,13cyclictest13171-21tail07:55:271
2222099148,5cyclictest9170-21sh10:23:470
2222099140,0cyclictest0-21swapper/009:21:060
311682130,1sleep015227-21diskmemload09:38:450
2222999139,3cyclictest14871-21perf11:45:003
2222999139,3cyclictest1149-21cat09:20:203
22229991310,2cyclictest9795-21awk10:05:003
22229991310,2cyclictest6200-21grep08:50:133
22229991310,2cyclictest5128-21grep07:40:183
22229991310,2cyclictest31961-21grep08:35:173
22229991310,2cyclictest3022-21ssh10:20:133
22229991310,2cyclictest29755-21cron11:15:003
22229991310,2cyclictest25884-21head09:35:193
22229991310,2cyclictest25619-21grep12:10:163
22229991310,2cyclictest24754-21cat07:15:013
2222999130,12cyclictest21760-21ssh11:29:373
2222999130,12cyclictest0-21swapper/310:56:333
22226991312,1cyclictest0-21swapper/210:15:432
22226991312,1cyclictest0-21swapper/207:50:132
22226991311,1cyclictest15227-21diskmemload11:55:112
2222699131,10cyclictest0-21swapper/210:29:332
22226991310,2cyclictest17480-21sed08:05:222
2222699130,2cyclictest0-21swapper/209:53:282
2222699130,13cyclictest0-21swapper/211:25:232
2222699130,12cyclictest0-21swapper/211:48:232
2222699130,12cyclictest0-21swapper/211:15:122
2222699130,12cyclictest0-21swapper/211:03:112
2222299136,6cyclictest19071-21cron10:30:001
2222299131,7cyclictest31391-21grep08:35:141
2222299131,1cyclictest29712-21ssh09:37:351
22222991310,2cyclictest7010-21grep11:00:271
2222299130,12cyclictest24885-21tr08:20:221
2222299130,12cyclictest0-21swapper/109:43:351
22220991310,2cyclictest32027-21head08:35:170
2222099130,12cyclictest31768-21sh07:30:010
13942130,1sleep30-21swapper/311:16:373
2222999129,2cyclictest8886-21awk07:50:003
2222999129,2cyclictest5878-21grep11:00:183
2222999129,2cyclictest5703-21cron08:50:013
2222999129,2cyclictest26937-21cron07:20:003
2222999129,2cyclictest2655-21cat09:40:243
2222999129,2cyclictest17673-21head09:10:253
2222999129,2cyclictest16329-21taskset10:27:533
2222999129,2cyclictest10320-21seq12:20:213
2222999128,2cyclictest340-21head11:55:153
2222999128,2cyclictest11239-21cron07:55:003
2222999127,4cyclictest25249-21ssh10:33:193
2222999123,2cyclictest0-21swapper/310:20:003
22229991211,1cyclictest0-21swapper/312:00:113
2222699129,2cyclictest4964-21ssh09:42:012
2222699129,2cyclictest32081-21ssh12:33:552
2222699129,2cyclictest27483-21ssh11:13:132
2222699129,2cyclictest15805-21ssh12:24:122
2222699129,2cyclictest10389-21ssh10:43:482
2222699128,3cyclictest18188-21tail12:25:172
2222699128,3cyclictest14050-21egrep08:00:132
2222699127,5cyclictest0-21swapper/209:27:592
2222699127,2cyclictest0-21swapper/209:46:172
22226991210,1cyclictest32-21ksoftirqd/211:30:272
22226991210,1cyclictest32073-21grep07:30:122
22226991210,1cyclictest29558-21grep08:30:172
22226991210,1cyclictest16589-21cut09:10:172
22226991210,1cyclictest1512-21tr08:40:142
2222699120,5cyclictest0-21swapper/211:21:572
2222699120,1cyclictest0-21swapper/210:13:132
2222299129,2cyclictest32203-21tail11:15:241
2222299129,2cyclictest3125-21wc10:00:201
2222299129,2cyclictest27902-21sed09:55:271
2222299129,2cyclictest1514-21grep12:15:151
2222299129,2cyclictest13902-21sed09:05:171
2222299128,4cyclictest16103-21ssh12:05:061
2222299128,3cyclictest11006-21tr09:00:141
2222299128,2cyclictest14149-21grep08:00:141
2222299127,5cyclictest38150irq/131-snd_hda10:17:591
2222299124,5cyclictest23355-21cron08:20:001
22222991210,2cyclictest25-21ksoftirqd/109:45:181
2222299120,2cyclictest0-21swapper/111:35:051
2222299120,12cyclictest0-21swapper/111:30:111
2222299120,12cyclictest0-21swapper/110:51:051
2222099129,2cyclictest7613-21grep08:50:250
2222099129,2cyclictest30053-21grep08:30:220
2222099129,2cyclictest24520-21cat08:20:180
2222099129,2cyclictest17669-21ssh10:47:580
2222099129,2cyclictest17669-21ssh10:47:570
2222099129,2cyclictest16077-21cut11:45:150
2222099129,2cyclictest14697-21ssh12:23:200
2222099129,2cyclictest13106-21cat11:05:000
2222099124,7cyclictest21923-21ssh11:10:120
2222099124,6cyclictest8317-21/usr/sbin/munin08:55:210
2222099121,7cyclictest15227-21diskmemload09:15:110
2222099121,1cyclictest0-21swapper/011:40:540
2222999118,2cyclictest620-21ssh12:15:103
2222999118,2cyclictest4032-21taskset07:39:153
2222999118,2cyclictest28144-21ssh10:35:143
2222999118,2cyclictest23070-21taskset08:16:263
2222999118,2cyclictest18929-21tr08:10:153
2222999118,2cyclictest17852-21taskset09:50:133
2222999118,2cyclictest17441-21ntpq08:05:213
2222999118,2cyclictest16035-21ssh12:05:033
2222999118,2cyclictest13484-21taskset07:58:243
2222999112,7cyclictest24518-21gltestperf11:50:163
2222999110,9cyclictest0-21swapper/310:11:023
2222999110,10cyclictest24465-21basename08:20:173
2222699119,2cyclictest13278-21chrt07:55:442
2222699119,1cyclictest7963-21sed07:45:212
2222699119,1cyclictest7851-21grep11:40:172
2222699119,1cyclictest29053-21sh09:37:042
2222699119,1cyclictest23611-21ssh12:09:412
2222699119,1cyclictest22718-21tr07:10:142
2222699119,1cyclictest206502sleep208:13:442
2222699119,1cyclictest14832-21cat11:05:202
2222699118,2cyclictest9110-21cut08:55:162
2222699118,2cyclictest8837-21rm12:00:182
2222699118,2cyclictest6663-21cat07:45:002
2222699118,2cyclictest6141-21ssh12:18:092
2222699118,2cyclictest31368-21ssh10:55:352
2222699118,2cyclictest27090-21ssh09:55:192
2222699118,2cyclictest26972-21basename07:20:002
2222699118,2cyclictest26950-21tail08:25:172
2222699118,2cyclictest2239-21cut07:35:162
2222699118,2cyclictest16649-21ssh09:30:122
2222699118,2cyclictest14352-21head09:05:222
2222299119,1cyclictest7015-21grep11:20:181
2222299118,2cyclictest8738-21seq07:48:251
2222299118,2cyclictest26526-21tr08:25:151
2222299118,2cyclictest22593-21ssh09:33:311
2222299118,2cyclictest20202-21ssh10:30:161
2222299118,2cyclictest15602-21ssh09:29:351
2222299118,2cyclictest11594-21ssh12:01:441
2222299117,3cyclictest9392-21kworker/1:111:07:351
2222299112,8cyclictest0-21swapper/107:25:121
2222299111,6cyclictest225-21systemd-journal08:10:001
2222299110,5cyclictest27882-21fschecks_time07:20:161
2222299110,10cyclictest29639-21basename08:30:181
2222099119,2cyclictest4227-21ssh12:16:370
2222099119,2cyclictest27354-21taskset11:32:280
2222099119,2cyclictest27259-21chrt08:25:200
2222099119,2cyclictest18720-21ssh09:11:120
2222099119,2cyclictest1519-21ssh11:36:040
2222099119,1cyclictest32626-21ssh10:56:350
2222099119,1cyclictest31094-21ssh12:13:510
2222099119,1cyclictest31065-21head11:15:150
2222099119,1cyclictest28656-21sed10:15:240
2222099119,1cyclictest26434-21head12:30:160
2222099119,1cyclictest25975-21basename09:55:140
2222099119,1cyclictest17626-21rm11:07:000
2222099119,1cyclictest12124-21ssh11:23:350
2222099119,1cyclictest11149-21grep09:45:260
2222099118,2cyclictest9426-21tr07:50:150
2222099118,2cyclictest4071-21expr08:45:140
2222099118,2cyclictest17820-21chrt12:05:220
2222099118,2cyclictest16304-21basename08:05:140
2222099118,2cyclictest14186-21tr08:00:130
2222099111,1cyclictest0-21swapper/010:27:050
2222099110,7cyclictest2841-21ntp_kernel_pll_12:35:210
2222099110,1cyclictest0-21swapper/008:15:140
2222099110,10cyclictest19935-21cat10:30:150
2222999108,1cyclictest8313-21seq08:55:083
2222999108,1cyclictest32503-21cat07:30:143
2222999108,1cyclictest3002-21chrt08:40:493
2222999108,1cyclictest10737-21apache_accesses09:00:113
2222999107,2cyclictest29336-21sh07:25:003
2222999107,2cyclictest26465-21tr08:25:143
2222999107,2cyclictest13899-21expr09:05:173
2222699108,1cyclictest9082sleep208:39:532
2222699108,1cyclictest77262sleep208:50:292
2222699108,1cyclictest256682sleep208:24:012
2222699108,1cyclictest24474-21expr09:15:142
2222699107,2cyclictest5204-21unixbench-2d08:45:252
2222699106,3cyclictest10885-21egrep09:00:132
2222699100,2cyclictest0-21swapper/208:15:142
2222299108,1cyclictest9369-21sshd08:55:181
2222299108,1cyclictest24273-21chrt07:11:231
2222299106,3cyclictest9906-21cut07:50:161
2222299106,2cyclictest6684-21wc07:45:001
2222099109,1cyclictest23455-21cat07:10:180
2222099109,1cyclictest14901-21taskset09:05:570
2222099108,2cyclictest19099-21cat08:10:140
2222099108,1cyclictest4575-21cpu07:40:150
2222099108,1cyclictest32155-21basename07:30:130
2222099108,1cyclictest11797-21cut07:55:150
2222099108,1cyclictest11084-21cat09:00:140
2222099107,2cyclictest28456-21grep07:20:210
2222099107,2cyclictest18846-21cut10:10:120
2222099107,2cyclictest1481-21grep08:40:140
2222099104,3cyclictest225-21systemd-journal07:49:590
222299996,2cyclictest30325-21grep07:25:173
222229996,2cyclictest3358-21mailstats07:35:231
222209997,1cyclictest266792sleep007:16:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional