You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 16:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat May 23, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
804799941,53cyclictest0-21swapper/009:15:020
277792920,9sleep3381ktimersoftd/312:34:083
804799906,55cyclictest0-21swapper/009:35:560
315102870,4sleep2804999cyclictest10:02:022
48812820,2sleep04882-21unixbench_singl12:05:310
215752780,2sleep00-21swapper/011:08:390
8047997734,20cyclictest0-21swapper/010:15:160
8047997620,23cyclictest19556-21aten_rbpower_vo11:45:170
8050997510,14cyclictest7850irq/56-dwc_otg09:55:163
804799745,50cyclictest0-21swapper/011:20:160
804799745,50cyclictest0-21swapper/011:20:160
804799731,56cyclictest13099-21fschecks_time09:40:190
804799731,56cyclictest13099-21fschecks_time09:40:190
804799712,54cyclictest3319-21latency_hist12:05:030
804799702,50cyclictest1106-21wget10:05:160
804799701,53cyclictest26422-21runrttasks09:55:320
805099695,12cyclictest0-21swapper/307:35:143
804899692,4cyclictest0-21swapper/109:40:131
804899692,4cyclictest0-21swapper/109:40:131
804799691,51cyclictest3824-21cron09:30:020
804799681,50cyclictest23525-21missed_timers11:10:390
804799678,48cyclictest0-21swapper/009:45:140
303552670,2sleep330356-21switchtime10:00:303
8047996637,19cyclictest0-21swapper/008:25:140
149982660,2sleep30-21swapper/309:42:103
149982660,2sleep30-21swapper/309:42:103
8047996439,11cyclictest0-21swapper/008:20:020
804799642,47cyclictest0-21swapper/007:35:290
804799641,51cyclictest0-21swapper/012:10:220
804799631,49cyclictest8094-21grep11:30:310
265082630,1sleep30-21swapper/311:53:263
8050996210,9cyclictest0-21swapper/311:00:143
804799622,42cyclictest0-21swapper/011:00:300
804799621,52cyclictest18233-21ps09:05:260
804799621,51cyclictest5494-21ssh09:30:310
804799621,48cyclictest25539-21grep12:30:300
121772620,2sleep10-21swapper/107:20:231
804899614,5cyclictest0-21swapper/108:50:141
804799612,47cyclictest0-21swapper/010:55:230
804799612,45cyclictest0-21swapper/012:35:130
804799611,50cyclictest14114-21fschecks_count08:55:200
804799611,47cyclictest15399-21cut07:30:160
804999604,11cyclictest8050irq/56-dwc_otg_11:10:152
804799608,33cyclictest0-21swapper/010:40:230
804799601,49cyclictest29892-21grep10:00:230
804799601,45cyclictest12261-21memory07:20:240
804799593,42cyclictest0-21swapper/010:50:120
804799591,49cyclictest0-21swapper/011:40:260
804999587,14cyclictest16920-21ssh09:45:142
804899584,12cyclictest7950irq/56-dwc_otg_11:35:151
804799582,46cyclictest3390-21missed_timers11:25:240
8047995820,20cyclictest1133-21runrttasks08:40:280
804799581,47cyclictest0-21swapper/007:40:430
804799581,45cyclictest29229-21rm11:17:270
49392580,1sleep00-21swapper/008:30:180
805099561,20cyclictest8050irq/56-dwc_otg_10:35:163
8049995614,19cyclictest11033-21ssh11:35:152
804899563,8cyclictest8050irq/56-dwc_otg_09:30:131
804799561,42cyclictest21316-21missed_timers07:45:250
804799561,39cyclictest28666-21latency09:20:230
8047995519,15cyclictest0-21swapper/009:15:050
226602550,2sleep00-21swapper/009:51:010
8050995415,10cyclictest7950irq/56-dwc_otg_10:45:153
804899542,15cyclictest22681-21sh11:10:141
8047995428,13cyclictest0-21swapper/010:20:160
804799541,46cyclictest0-21swapper/011:50:240
804799541,39cyclictest31535-21cron12:00:020
8047995317,22cyclictest16960-21ssh12:20:270
8047995317,22cyclictest16960-21ssh12:20:270
804799531,41cyclictest0-21swapper/007:15:130
804799531,38cyclictest0-21swapper/011:35:590
804799531,38cyclictest0-21swapper/010:11:190
804799531,38cyclictest0-21swapper/010:11:190
804899522,9cyclictest7950irq/56-dwc_otg_12:00:141
8047995127,12cyclictest7950irq/56-dwc_otg_08:50:020
804799511,39cyclictest0-21swapper/008:05:270
804799501,34cyclictest14636-21rm12:18:040
8047995013,14cyclictest7505-21ls08:35:310
8047995010,19cyclictest30414-21missed_timers08:10:250
804899493,4cyclictest7850irq/56-dwc_otg08:55:141
804799491,33cyclictest24854-21if_eth007:55:230
804799487,28cyclictest9-21ksoftirqd/010:35:130
804799482,7cyclictest9250-21unixbench_singl07:10:310
804799482,7cyclictest9250-21unixbench_singl07:10:310
804799481,7cyclictest21249-21ssh10:29:360
805099477,25cyclictest0-21swapper/307:55:083
804799471,5cyclictest2194-21grep08:20:300
804799471,5cyclictest2194-21grep08:20:300
8047994711,5cyclictest0-21swapper/010:30:090
40432470,5sleep2804999cyclictest12:05:212
804899468,12cyclictest0-21swapper/110:20:151
804899466,14cyclictest8050irq/56-dwc_otg_10:30:331
8047994623,2cyclictest0-21swapper/012:25:130
8047994620,5cyclictest7950irq/56-dwc_otg_10:45:160
805099457,23cyclictest0-21swapper/309:15:093
8050994534,7cyclictest381ktimersoftd/310:30:173
804999457,23cyclictest0-21swapper/207:50:162
804999457,23cyclictest0-21swapper/207:50:162
804999456,25cyclictest0-21swapper/208:40:082
8049994514,7cyclictest20672-21unixbench_multi09:10:322
804799451,26cyclictest0-21swapper/008:00:290
74882452,36sleep00-21swapper/007:06:160
805099443,8cyclictest7950irq/56-dwc_otg_08:40:153
8050994424,15cyclictest8050irq/56-dwc_otg_10:50:153
804999444,6cyclictest8050irq/56-dwc_otg_12:20:312
804999444,6cyclictest8050irq/56-dwc_otg_12:20:312
8049994420,6cyclictest21118-21sh10:29:212
804899447,25cyclictest0-21swapper/111:20:161
804899447,25cyclictest0-21swapper/111:20:161
8047994417,4cyclictest9550irq/80-mmc009:05:030
8047994417,4cyclictest9550irq/80-mmc009:05:030
804799441,33cyclictest0-21swapper/008:54:170
805099437,23cyclictest0-21swapper/308:15:163
216102430,4sleep121606-21sh10:30:031
805099427,23cyclictest0-21swapper/308:50:163
805099427,22cyclictest0-21swapper/309:45:163
805099423,4cyclictest0-21swapper/311:55:163
805099421,9cyclictest18058-21fschecks_time10:25:213
805099421,8cyclictest10470-21ls08:45:173
804999423,14cyclictest0-21swapper/208:25:142
8048994213,8cyclictest3183-21unixbench_multi10:45:321
804899421,28cyclictest141-21systemd-journal11:44:411
804799421,23cyclictest13807-21fschecks_time07:25:210
804799421,14cyclictest0-21swapper/007:55:030
804799421,14cyclictest0-21swapper/007:55:030
152392420,1sleep30-21swapper/307:30:033
111432420,2sleep10-21swapper/112:14:181
805099412,16cyclictest7850irq/56-dwc_otg08:35:153
805099411,10cyclictest39-21ksoftirqd/307:45:143
804999414,23cyclictest0-21swapper/207:15:162
804899411,9cyclictest771-21apt08:20:021
804899411,5cyclictest1165-21snmpd11:54:431
30582410,1sleep10-21swapper/110:06:441
304902410,1sleep30-21swapper/311:19:453
295792410,1sleep10-21swapper/111:18:061
805099407,25cyclictest0-21swapper/308:25:163
805099407,22cyclictest0-21swapper/312:35:163
804999407,19cyclictest0-21swapper/209:05:152
804999407,13cyclictest0-21swapper/208:50:142
805099397,8cyclictest7850irq/56-dwc_otg10:20:163
805099396,9cyclictest37-21rcuc/307:50:223
805099396,9cyclictest37-21rcuc/307:50:223
8050993918,14cyclictest0-21swapper/308:10:173
804999396,8cyclictest0-21swapper/212:00:142
804999393,8cyclictest0-21swapper/208:10:142
804899395,21cyclictest8050irq/56-dwc_otg_10:55:171
8048993921,8cyclictest23760-21/usr/sbin/munin09:15:251
8048993921,4cyclictest7169-21fschecks_count11:30:201
8048993918,10cyclictest24447-21sh09:54:161
8048993914,5cyclictest27328-21grep08:00:311
805099386,5cyclictest7850irq/56-dwc_otg12:14:473
805099383,27cyclictest8050irq/56-dwc_otg_11:45:143
805099382,4cyclictest7850irq/56-dwc_otg11:30:153
805099381,25cyclictest20710-21ls12:25:203
804999386,6cyclictest7950irq/56-dwc_otg_11:45:202
8049993819,12cyclictest3260-21rm10:45:322
8049993817,13cyclictest26910-21sh10:35:302
8049993816,6cyclictest25508-21timerandwakeup12:30:302
804899388,16cyclictest0-21swapper/107:35:141
8048993818,5cyclictest30695-21switchtime08:10:301
804899381,5cyclictest19397-21/usr/sbin/munin09:10:201
29692380,5sleep2804999cyclictest11:25:192
805099378,9cyclictest9531-21cpu10:15:183
805099377,14cyclictest8050irq/56-dwc_otg_12:10:023
805099375,6cyclictest0-21swapper/309:50:153
8050993717,4cyclictest1133-21runrttasks12:04:263
805099371,13cyclictest7950irq/56-dwc_otg_09:40:033
804999373,10cyclictest0-21swapper/208:55:142
8049993716,4cyclictest4732-21/usr/sbin/munin08:30:242
8049993710,20cyclictest9772-21grep09:35:322
804899376,7cyclictest7950irq/56-dwc_otg_09:35:321
804999367,6cyclictest7950irq/56-dwc_otg_10:15:162
8049993612,6cyclictest32111-21sh10:41:322
804899365,5cyclictest28307-21ssh12:35:021
804899363,8cyclictest9624-21grep08:40:311
8048993611,9cyclictest17586-21rm11:03:561
8048993611,5cyclictest31508-21grep10:40:321
805099354,6cyclictest7850irq/56-dwc_otg07:40:253
805099351,23cyclictest3671-21switchtime11:25:293
8050993510,13cyclictest37-21rcuc/309:04:113
8050993510,13cyclictest37-21rcuc/309:04:113
804999359,4cyclictest0-21swapper/211:04:362
804999358,15cyclictest0-21swapper/210:30:092
804999357,3cyclictest0-21swapper/209:27:012
804999355,19cyclictest27764-21ssh11:55:172
804999353,27cyclictest8050irq/56-dwc_otg_09:30:142
8049993518,5cyclictest268372chrt11:54:022
8049993515,12cyclictest14180-21sh12:17:142
804999351,24cyclictest10518-21ssh10:55:202
804899356,4cyclictest7850irq/56-dwc_otg10:50:211
804899355,6cyclictest7850irq/56-dwc_otg07:35:021
804899354,6cyclictest8050irq/56-dwc_otg_10:40:031
804899352,14cyclictest7950irq/56-dwc_otg_12:30:021
804899351,4cyclictest20141-21latency_hist12:25:031
804899351,4cyclictest20141-21latency_hist12:25:031
804899351,20cyclictest19058-21diskmemload10:10:191
804899351,20cyclictest19058-21diskmemload10:10:191
8048993510,6cyclictest28989-21/usr/sbin/munin09:20:281
8048993510,3cyclictest0-21swapper/109:55:221
65472350,1sleep30-21swapper/309:32:203
805099341,8cyclictest22744-21aten_rbpower_en11:10:173
805099341,7cyclictest7950irq/56-dwc_otg_09:20:303
804999348,18cyclictest14599-21ssh09:41:272
804999348,18cyclictest14599-21ssh09:41:272
8049993421,5cyclictest772-21latency_hist08:20:022
804999341,7cyclictest31021-21ssh11:20:162
804999341,7cyclictest31021-21ssh11:20:162
8049993417,13cyclictest0-21swapper/211:15:252
8049993412,15cyclictest32008-21awk09:25:022
804899346,13cyclictest8050irq/56-dwc_otg_07:50:021
804899345,6cyclictest7950irq/56-dwc_otg_11:25:311
804899344,5cyclictest7850irq/56-dwc_otg12:05:141
8048993417,9cyclictest17473-21/usr/sbin/munin09:05:171
8048993411,5cyclictest0-21swapper/107:50:301
8048993411,5cyclictest0-21swapper/107:50:301
8048993410,9cyclictest0-21swapper/109:46:001
72922346,15sleep17411-21uptime07:05:321
277422341,30sleep20-21swapper/209:57:442
805099339,13cyclictest0-21swapper/311:39:393
805099337,14cyclictest0-21swapper/310:05:163
805099334,4cyclictest8050irq/56-dwc_otg_12:15:143
805099334,11cyclictest7950irq/56-dwc_otg_08:21:173
805099334,11cyclictest7950irq/56-dwc_otg_08:21:173
805099332,6cyclictest7850irq/56-dwc_otg10:40:253
805099332,6cyclictest7850irq/56-dwc_otg08:00:173
8050993323,8cyclictest8050irq/56-dwc_otg_11:20:463
8050993323,8cyclictest8050irq/56-dwc_otg_11:20:463
804999334,6cyclictest1415-21fschecks_time10:05:202
804999333,8cyclictest0-21swapper/212:35:142
804899336,10cyclictest8050irq/56-dwc_otg_08:40:021
804899335,6cyclictest1133-21runrttasks11:56:211
8048993320,7cyclictest15064-21idleruntime-cro07:30:021
804899331,8cyclictest5864-21grep08:30:311
8048993318,4cyclictest19352-21ssh11:45:081
8048993314,6cyclictest0-21swapper/110:00:191
159362331,5sleep215955-21ntp_states11:40:262
805099321,22cyclictest3838-21latency_hist09:30:023
8050993212,11cyclictest10858-21sh10:55:233
8050993211,6cyclictest39-21ksoftirqd/309:15:033
804999329,14cyclictest29-21rcuc/211:32:482
804999327,16cyclictest0-21swapper/209:50:172
804999327,15cyclictest0-21swapper/209:00:152
804999327,15cyclictest0-21swapper/209:00:152
804999327,15cyclictest0-21swapper/207:30:162
804999325,11cyclictest5229-21ssh10:10:142
804999325,11cyclictest5229-21ssh10:10:142
804999321,3cyclictest7950irq/56-dwc_otg_11:05:132
804999321,16cyclictest8050irq/56-dwc_otg_10:20:222
8049993211,4cyclictest0-21swapper/210:51:522
804899329,6cyclictest20039-21grep07:40:311
804899327,13cyclictest0-21swapper/111:05:281
804899326,13cyclictest8050irq/56-dwc_otg_10:15:181
804899323,6cyclictest8050irq/56-dwc_otg_07:15:201
804899322,7cyclictest10542-21fschecks_count08:45:201
8048993221,8cyclictest32735-21fschecks_time09:25:211
805099317,6cyclictest39-21ksoftirqd/308:06:483
805099315,5cyclictest7850irq/56-dwc_otg11:40:273
805099312,16cyclictest0-21swapper/307:22:383
805099311,14cyclictest8050irq/56-dwc_otg_07:15:203
804999315,5cyclictest0-21swapper/207:25:022
804999312,7cyclictest8050irq/56-dwc_otg_08:05:322
804999311,6cyclictest8050irq/56-dwc_otg_08:35:182
8049993113,12cyclictest139742sleep207:25:242
804999311,21cyclictest0-21swapper/208:03:462
804999311,20cyclictest0-21swapper/207:45:162
8049993110,4cyclictest0-21swapper/212:30:022
804899318,6cyclictest16798-21grep09:00:301
804899318,6cyclictest16798-21grep09:00:301
804899314,5cyclictest7950irq/56-dwc_otg_12:20:021
804899311,6cyclictest8882-21ps07:10:271
804899311,6cyclictest8882-21ps07:10:271
282552310,1sleep10-21swapper/108:05:191
805099309,4cyclictest0-21swapper/310:10:103
805099309,4cyclictest0-21swapper/310:10:103
805099303,24cyclictest8050irq/56-dwc_otg_12:20:133
805099303,24cyclictest8050irq/56-dwc_otg_12:20:133
805099302,2cyclictest0-21swapper/307:30:303
805099301,3cyclictest0-21swapper/307:10:163
805099301,3cyclictest0-21swapper/307:10:163
805099301,20cyclictest0-21swapper/308:32:173
804999302,19cyclictest0-21swapper/207:40:252
804999301,7cyclictest8369-21ssh12:10:222
8049993014,9cyclictest8424-21idleruntime07:10:202
8049993014,9cyclictest8424-21idleruntime07:10:202
804999301,22cyclictest0-21swapper/209:16:182
8049993011,12cyclictest301ktimersoftd/207:35:162
804899309,5cyclictest7850irq/56-dwc_otg07:55:291
804899307,4cyclictest0-21swapper/112:35:171
804899303,6cyclictest8050irq/56-dwc_otg_08:30:031
805099299,11cyclictest18544-21ssh11:05:133
804999291,7cyclictest2602-21munin-run08:25:032
804999291,7cyclictest2602-21munin-run08:25:032
805099284,5cyclictest8050irq/56-dwc_otg_09:00:033
804999288,7cyclictest8050irq/56-dwc_otg_08:45:152
804999286,14cyclictest7850irq/56-dwc_otg07:55:172
804899285,11cyclictest7950irq/56-dwc_otg_08:21:391
804899285,11cyclictest7950irq/56-dwc_otg_08:21:391
74342273,14sleep30-21swapper/307:05:343
62302275,13sleep20-21swapper/207:05:132
8050992618,6cyclictest37-21rcuc/309:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional