You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 06:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu May 07, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2333121030,6sleep223296-21ssh08:20:562
2333121030,6sleep223296-21ssh08:20:562
1605099896,75cyclictest0-21swapper/006:16:080
141452880,2sleep20-21swapper/205:55:552
1605099847,24cyclictest0-21swapper/005:06:070
194902790,2sleep20-21swapper/207:36:392
318412760,2sleep10-21swapper/109:10:231
1605099737,58cyclictest0-21swapper/009:01:080
317182720,2sleep00-21swapper/009:48:470
269992720,5sleep11605199cyclictest06:30:561
99732700,1sleep10-21swapper/106:48:071
1605199692,14cyclictest0-21swapper/106:21:091
1605099698,44cyclictest5881-21ssh06:43:240
1605099688,50cyclictest0-21swapper/006:51:180
1605099682,53cyclictest11326-21ssh07:27:180
1605099682,53cyclictest11326-21ssh07:27:180
257212670,2sleep20-21swapper/206:26:142
1605099674,45cyclictest23494-21wget06:21:090
251252660,1sleep30-21swapper/307:44:013
1605099664,44cyclictest12852-21timerandwakeup08:46:230
1605099662,49cyclictest23457-21unixbench-2d07:41:200
16050996611,40cyclictest17987-21aten_rbpower_vo04:36:080
18122650,2sleep31605399cyclictest06:38:473
1605099658,48cyclictest0-21swapper/006:32:250
1605099657,40cyclictest23804-21fschecks_count08:21:120
1605099651,50cyclictest27891-21/usr/sbin/munin08:26:120
71382640,3sleep013-21rcuc/009:19:200
1605099647,41cyclictest12797-21fschecks_count05:51:120
1605099646,43cyclictest32573-21grep05:16:110
1605099642,49cyclictest10592-21latency_hist05:45:550
1605099642,42cyclictest9-21ksoftirqd/007:06:230
16050996411,36cyclictest1067-21aten_rbpower_vo07:16:100
1605099638,38cyclictest8780-21chrt09:21:130
1605099629,38cyclictest24241-21ssh07:05:270
1605099624,47cyclictest23006-21ssh08:20:500
1605099624,47cyclictest23006-21ssh08:20:500
16050996210,35cyclictest31263-21sh09:09:190
16050996210,35cyclictest10725-21latency_hist08:05:550
144422620,9sleep3381ktimersoftd/308:48:513
57702610,6sleep01605099cyclictest09:56:160
16052996134,9cyclictest7815-21ls08:01:202
1605099615,41cyclictest3106-21runrttasks05:24:180
1605099614,41cyclictest0-21swapper/009:51:090
1605099613,47cyclictest7640-21chrt06:46:050
16051996020,7cyclictest28781-21munin-run07:10:551
1605099608,41cyclictest0-21swapper/008:43:590
1605099607,38cyclictest0-21swapper/008:57:510
1605099607,38cyclictest0-21swapper/008:57:510
16050996037,13cyclictest7950irq/56-dwc_otg_09:41:200
1605099603,46cyclictest16279-21/usr/sbin/munin06:56:120
1605099603,46cyclictest16279-21/usr/sbin/munin06:56:120
1605099599,33cyclictest6500-21ssh08:00:550
1605099599,33cyclictest6500-21ssh08:00:550
1605099598,37cyclictest1089-21ssh08:32:100
1605099598,37cyclictest1089-21ssh08:32:100
1605099597,36cyclictest23861-21rm09:39:330
16050995930,14cyclictest7850irq/56-dwc_otg_06:30:550
16050995916,31cyclictest17713-21latency_hist06:05:550
1605099587,31cyclictest18380-21ssh09:32:090
16050995839,5cyclictest25391-21fschecks_time04:56:120
1605099583,40cyclictest2200-21ssh07:55:430
16051995718,20cyclictest14881-21ssh08:49:401
1605099578,37cyclictest8541-21rm07:24:530
1605099576,37cyclictest19793-21basename04:41:080
1605099576,37cyclictest19793-21basename04:41:080
16050995734,5cyclictest31642-21sed05:11:200
16050995716,24cyclictest28472-21ls05:05:540
16052995634,10cyclictest17873-21apt-get04:36:072
16052995633,11cyclictest1832-21ssh09:12:142
16052995626,10cyclictest7950irq/56-dwc_otg_09:01:162
1605099565,37cyclictest0-21swapper/006:13:010
16050995644,4cyclictest7850irq/56-dwc_otg_08:06:220
16052995533,11cyclictest12702-21ssh06:51:142
16052995529,10cyclictest7245-21ssh06:45:472
16052995526,11cyclictest31567-21ssh07:13:212
16051995521,18cyclictest30450-21ssh09:46:281
1605099554,37cyclictest23085-21latency_hist04:50:540
1605099554,37cyclictest23085-21latency_hist04:50:540
1605099553,28cyclictest0-21swapper/006:06:230
1605099552,38cyclictest0-21swapper/005:31:350
1605099551,41cyclictest3633-21ssh09:15:300
16052995429,9cyclictest18427-21ssh09:32:142
16052995429,10cyclictest6588-21ssh09:18:202
16051995418,16cyclictest945-21dpkg07:16:061
1605099549,35cyclictest20542-21ssh07:38:280
1605099546,37cyclictest7193-21sh08:40:440
16050995436,4cyclictest7750irq/56-dwc_otg06:36:130
1605099542,29cyclictest0-21swapper/005:26:240
16050995417,26cyclictest0-21swapper/005:36:090
1605099540,36cyclictest18482-21ssh08:15:080
16052995329,10cyclictest561-21ssh07:52:532
16052995327,12cyclictest25164-21ssh07:44:062
16052995326,10cyclictest7950irq/56-dwc_otg_06:37:352
16052995325,16cyclictest7950irq/56-dwc_otg_05:57:062
16052995324,11cyclictest8619-21ssh09:21:122
16051995313,14cyclictest23009-21/usr/sbin/munin07:41:151
1605099538,28cyclictest10786-21aten_rbpower_en05:46:090
1605099537,30cyclictest19204-21ssh08:55:000
1605099531,29cyclictest17162-21unixbench_multi04:31:220
16052995231,10cyclictest1090-21ssh08:32:102
16052995231,10cyclictest1090-21ssh08:32:102
16052995229,10cyclictest17535-21ssh07:35:422
16052995228,15cyclictest30554-21ssh06:34:472
16052995225,10cyclictest7850irq/56-dwc_otg_07:21:472
16052995224,10cyclictest12340-21ssh08:06:312
16052995223,9cyclictest7950irq/56-dwc_otg_07:30:102
16052995223,9cyclictest7950irq/56-dwc_otg_07:30:102
16052995222,9cyclictest7850irq/56-dwc_otg_09:29:502
16052995222,10cyclictest10207-21ssh06:48:322
16051995227,8cyclictest24191-21apt-get09:01:061
1605099527,32cyclictest0-21swapper/009:26:100
16050995237,5cyclictest0-21swapper/007:11:070
16050995224,11cyclictest24436-21unixbench_singl04:51:230
16052995137,11cyclictest2656-21ssh07:56:052
16052995137,11cyclictest2656-21ssh07:56:052
16052995130,10cyclictest23175-21ssh09:38:192
16052995128,10cyclictest22425-21ssh08:58:182
16052995128,10cyclictest22425-21ssh08:58:182
16052995126,9cyclictest7950irq/56-dwc_otg_08:44:042
16052995126,8cyclictest25193-21ssh09:41:112
16052995121,9cyclictest7950irq/56-dwc_otg_09:58:152
16051995118,19cyclictest2051-21rm09:12:381
16051995117,18cyclictest0-21swapper/106:16:071
16051995115,22cyclictest17917-21ssh06:56:531
16051995115,22cyclictest17917-21ssh06:56:531
1605099512,24cyclictest30576-21latency_hist07:50:550
16050995120,17cyclictest0-21swapper/005:56:060
1605099511,6cyclictest13848-21ssh07:31:090
16052995035,12cyclictest4113-21ls08:36:162
16052995029,10cyclictest24003-21ssh07:05:032
16052995028,10cyclictest29729-21ssh09:06:322
16052995024,11cyclictest26247-21ssh08:24:002
16052995020,18cyclictest7950irq/56-dwc_otg_05:46:192
16051995019,16cyclictest13453-21rm06:51:341
16051995015,17cyclictest23133-21ssh09:38:131
16051995012,11cyclictest17896-21ssh08:52:361
1605399491,20cyclictest0-21swapper/309:56:073
16052994927,11cyclictest18463-21ssh08:15:062
16052994927,10cyclictest28200-21ssh07:46:562
16052994927,10cyclictest19455-21ssh06:59:352
16052994927,10cyclictest19455-21ssh06:59:352
16052994926,15cyclictest7950irq/56-dwc_otg_05:43:362
16052994925,13cyclictest2555-21ssh09:52:112
16052994925,11cyclictest14899-21ssh08:49:422
16052994923,11cyclictest30668-21ssh09:46:512
16051994917,13cyclictest17489-21ssh07:35:371
16051994914,14cyclictest21586-21rm08:18:141
16051994914,14cyclictest21586-21rm08:18:141
16051994914,11cyclictest15443-21ssh09:29:221
16051994913,11cyclictest30925-21ssh06:35:261
1605399487,26cyclictest15109-21ssh08:11:083
16053994822,16cyclictest381ktimersoftd/308:16:093
16053994822,16cyclictest381ktimersoftd/308:16:093
16052994836,8cyclictest19691-21apt-get04:41:052
16052994836,8cyclictest19691-21apt-get04:41:052
16052994828,10cyclictest29010-21ssh08:26:292
16051994825,8cyclictest7629-21apt-get08:41:051
16051994815,15cyclictest12545-21sh08:06:541
16051994812,17cyclictest26828-21diskmemload07:02:421
16051994811,7cyclictest7850irq/56-dwc_otg_08:21:141
1605399472,38cyclictest0-21swapper/309:41:093
16052994727,8cyclictest2298-21ssh07:16:262
16052994727,11cyclictest27171-21ssh07:08:082
1605199475,27cyclictest0-21swapper/108:38:181
16051994719,11cyclictest20122-21if_wlan004:41:131
16051994719,11cyclictest20122-21if_wlan004:41:131
16051994714,20cyclictest6543-21ssh09:18:161
16051994714,17cyclictest8364-21ssh09:21:071
156932473,35sleep00-21swapper/004:29:550
16052994637,7cyclictest481-21ls05:16:152
1605199467,13cyclictest917-21snmpd07:30:071
1605199467,13cyclictest917-21snmpd07:30:071
16051994617,15cyclictest7214-21idleruntime08:01:121
16051994617,11cyclictest2544-21ssh09:52:101
16051994617,10cyclictest10539-21latency_hist05:45:551
16051994615,18cyclictest17388-21taskset06:02:011
16051994611,14cyclictest29449-21ssh08:27:171
16051994611,11cyclictest25196-21df09:41:111
1605399458,23cyclictest0-21swapper/308:26:093
1605399456,25cyclictest0-21swapper/306:51:013
16052994531,11cyclictest6272-21ls05:31:212
16052994524,10cyclictest7750irq/56-dwc_otg06:21:132
1605399447,23cyclictest0-21swapper/305:56:093
1605299445,9cyclictest7950irq/56-dwc_otg_05:06:052
16052994430,11cyclictest16560-21missed_timers08:51:162
16052994422,6cyclictest7950irq/56-dwc_otg_06:16:072
16052994421,10cyclictest7850irq/56-dwc_otg_05:11:142
1605199442,24cyclictest0-21swapper/109:35:311
16051994418,15cyclictest17607-21seq04:35:471
1605199441,6cyclictest5104-21ssh09:56:081
16051994413,7cyclictest31794-21ssh07:13:441
16051994413,12cyclictest30036-21ssh07:50:081
16051994410,7cyclictest403-21ls05:16:141
1605399435,24cyclictest0-21swapper/305:41:083
16053994324,5cyclictest20548-21/usr/sbin/munin07:01:123
1605399432,26cyclictest0-21swapper/305:51:083
1605299435,9cyclictest7950irq/56-dwc_otg_04:46:062
1605299435,9cyclictest7950irq/56-dwc_otg_04:46:062
16052994333,7cyclictest4403-21ls05:26:222
16052994330,11cyclictest17193-21ls06:01:202
1605199439,20cyclictest7950irq/56-dwc_otg_07:55:451
1605199436,22cyclictest1336-21ssh08:32:371
1605199436,22cyclictest1336-21ssh08:32:371
1605199433,25cyclictest0-21swapper/105:06:071
16051994317,13cyclictest5104-21rm07:58:261
16051994317,13cyclictest5104-21rm07:58:261
16051994312,28cyclictest106650ssh06:37:311
16051994312,12cyclictest32126-21kworker/u8:407:40:261
16051994311,18cyclictest18702-21ssh08:15:321
66812420,1sleep30-21swapper/307:21:393
1605399427,22cyclictest0-21swapper/306:41:083
1605399426,22cyclictest0-21swapper/304:31:073
16053994214,4cyclictest0-21swapper/307:55:553
16052994233,7cyclictest26015-21ls04:56:202
1605199427,16cyclictest7850irq/56-dwc_otg_06:10:401
1605199426,24cyclictest0-21swapper/105:21:071
16051994217,11cyclictest13039-21sed05:51:141
16051994214,16cyclictest0-21swapper/106:44:161
1605399417,25cyclictest0-21swapper/306:31:083
16052994132,7cyclictest23294-21apt-get04:51:082
16052994127,11cyclictest7950irq/56-dwc_otg_04:33:292
16052994123,10cyclictest7950irq/56-dwc_otg_05:37:532
1605199419,11cyclictest28204-21sshd05:02:421
16051994112,11cyclictest18075-21df04:36:091
16051994110,19cyclictest6902-21ssh07:22:021
16051994110,10cyclictest9550irq/80-mmc005:30:561
16051994110,10cyclictest7750irq/56-dwc_otg05:11:141
1605399407,23cyclictest0-21swapper/306:21:093
1605399407,21cyclictest0-21swapper/307:46:083
1605399407,21cyclictest0-21swapper/305:06:073
1605399406,15cyclictest0-21swapper/309:36:083
16051994011,18cyclictest15789-21runrttasks05:59:421
1605399396,20cyclictest32511-21ssh09:11:083
16053993913,12cyclictest381ktimersoftd/308:01:163
16052993930,7cyclictest27958-21ls05:01:222
16052993930,7cyclictest20664-21ls06:11:212
16052993929,7cyclictest2239-21ls05:21:142
16052993925,11cyclictest18612-21ntp_kernel_err06:06:172
1605199397,7cyclictest0-21swapper/104:51:231
1605199392,11cyclictest7850irq/56-dwc_otg_08:58:011
1605199392,11cyclictest7850irq/56-dwc_otg_08:58:011
16051993916,12cyclictest9550irq/80-mmc006:11:061
16051993915,7cyclictest11288-21memory05:46:151
16051993910,11cyclictest8854-21cat05:40:551
245202380,2sleep3381ktimersoftd/308:21:213
1605399389,12cyclictest7850irq/56-dwc_otg_07:00:553
1605399389,12cyclictest7850irq/56-dwc_otg_07:00:553
1605399381,25cyclictest23789-21ls04:51:153
1605399381,24cyclictest381ktimersoftd/308:35:213
1605399381,24cyclictest381ktimersoftd/308:35:213
184142370,2sleep30-21swapper/306:06:143
1605399376,20cyclictest7741-21grep06:46:083
1605399373,6cyclictest37-21rcuc/304:41:093
1605399373,6cyclictest37-21rcuc/304:41:093
16053993721,4cyclictest18266-21sh08:53:173
16053993720,5cyclictest29076-21/usr/sbin/munin09:46:123
16053993718,12cyclictest381ktimersoftd/307:58:133
16053993718,12cyclictest381ktimersoftd/307:58:133
1605399367,17cyclictest381ktimersoftd/307:16:133
1605399366,15cyclictest381ktimersoftd/309:21:173
1605399365,14cyclictest37-21rcuc/305:16:103
16053993619,10cyclictest14386-21ssh09:27:273
16053993614,10cyclictest381ktimersoftd/307:10:403
1605199368,14cyclictest22932-21seq04:49:351
1605199368,14cyclictest22932-21seq04:49:351
150322366,10sleep10-21swapper/104:26:201
1605399356,21cyclictest0-21swapper/307:29:113
1605399356,21cyclictest0-21swapper/307:29:113
1605399354,7cyclictest381ktimersoftd/305:11:073
1605399354,16cyclictest37-21rcuc/309:08:423
1605399353,18cyclictest381ktimersoftd/304:36:123
16053993520,8cyclictest381ktimersoftd/307:36:073
1605399348,6cyclictest381ktimersoftd/309:35:053
1605399348,18cyclictest39-21ksoftirqd/306:02:103
1605399344,9cyclictest37-21rcuc/309:52:123
1605399344,12cyclictest37-21rcuc/308:41:033
16053993417,9cyclictest381ktimersoftd/307:31:063
16053993416,8cyclictest381ktimersoftd/306:16:073
1605399341,22cyclictest28650-21ls05:05:553
1605399341,18cyclictest0-21swapper/309:01:233
1605399337,7cyclictest381ktimersoftd/308:57:553
1605399337,7cyclictest381ktimersoftd/308:57:553
1605399333,7cyclictest37-21rcuc/307:13:423
1605399331,16cyclictest37-21rcuc/309:20:563
1605399331,10cyclictest39-21ksoftirqd/305:46:153
16053993310,4cyclictest0-21swapper/308:36:223
16051993318,12cyclictest25521-21awk04:56:131
16053993213,11cyclictest908-21runrttasks08:06:273
1605399321,19cyclictest0-21swapper/306:26:063
1605199328,9cyclictest0-21swapper/105:32:441
1605399311,5cyclictest381ktimersoftd/305:38:043
1605399311,24cyclictest6245-21ls05:31:213
1605399311,19cyclictest381ktimersoftd/305:22:023
1605399311,18cyclictest0-21swapper/306:11:233
156732313,22sleep20-21swapper/204:29:412
149252318,12sleep30-21swapper/304:26:183
1605399304,15cyclictest39-21ksoftirqd/304:46:553
1605399304,15cyclictest39-21ksoftirqd/304:46:553
1605399303,18cyclictest7950irq/56-dwc_otg_04:57:223
1605399285,12cyclictest381ktimersoftd/305:27:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional