You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 14:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Jul 10, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3251621850,6sleep31823499cyclictest10:34:393
3251621850,6sleep31823499cyclictest10:34:393
18229991091,88cyclictest0-21swapper/010:15:330
182299910731,61cyclictest8767-21ssh12:02:540
18232991064,23cyclictest7950irq/56-dwc_otg_12:35:171
18232991064,23cyclictest7950irq/56-dwc_otg_12:35:171
182299910329,61cyclictest13977-21grep08:25:460
182299910329,61cyclictest13977-21grep08:25:460
182299910227,51cyclictest24271-21tune2fs08:55:350
182299910227,51cyclictest24271-21tune2fs08:55:350
182299910065,14cyclictest0-21swapper/012:35:180
182299910065,14cyclictest0-21swapper/012:35:180
182299910028,52cyclictest18098-21timerandwakeup09:35:450
18229999530,48cyclictest31577-21/usr/sbin/munin07:45:400
18229999430,46cyclictest12252-21awk10:10:150
18229999429,45cyclictest11665-21grep12:05:440
18229999328,50cyclictest27945-21sed12:25:430
18229999327,48cyclictest0-21swapper/009:20:360
18229999327,43cyclictest14278-21unixbench_singl09:30:460
18229999227,45cyclictest10968-21grep08:20:160
18229999131,48cyclictest30959-21grep11:10:440
18229999023,52cyclictest0-21swapper/007:40:370
18229998928,42cyclictest0-21swapper/011:05:430
18229998832,11cyclictest14883-21tune2fs12:10:320
18229998827,42cyclictest0-21swapper/011:45:450
18229998824,47cyclictest0-21swapper/010:05:160
18229998732,34cyclictest15075-21ssh10:12:530
18229998726,43cyclictest22148-21/usr/sbin/munin11:40:260
18229998726,33cyclictest0-21swapper/009:49:030
18229998724,48cyclictest8805-21chrt08:12:290
18229998626,46cyclictest0-21swapper/011:15:380
18229998624,40cyclictest6637-21runrttasks10:41:320
163292860,9sleep3381ktimersoftd/311:32:493
18229998529,39cyclictest6732-21latency_hist12:40:160
18229998526,41cyclictest0-21swapper/007:32:210
18229998524,46cyclictest0-21swapper/009:27:460
18229998433,28cyclictest9581-21ssh11:25:200
18229998428,44cyclictest0-21swapper/009:14:230
18229998425,45cyclictest0-21swapper/010:34:030
18229998425,45cyclictest0-21swapper/010:34:030
18229998330,40cyclictest707-21/usr/sbin/munin10:35:350
18229998231,40cyclictest25173-21ssh10:25:310
18229998231,39cyclictest23763-21chrt11:02:380
18229998228,40cyclictest10885-21rm10:46:460
18229998228,26cyclictest3004-21ssh09:17:020
18229998226,42cyclictest30815-21ssh09:51:420
18229998226,26cyclictest3910-21ssh09:59:390
18229998224,11cyclictest22502-21modprobe12:20:150
18229998127,43cyclictest32201-21sh11:51:330
18229998126,39cyclictest11705-21grep08:20:380
18229998125,41cyclictest0-21swapper/010:23:230
18229998123,44cyclictest7298-21sh11:21:340
18229998026,43cyclictest16881-21/usr/sbin/munin08:35:340
18229998026,41cyclictest0-21swapper/009:40:360
18229998023,12cyclictest28961-21latency_hist07:40:160
18229998021,33cyclictest13936-21ssh10:50:390
18229998020,46cyclictest0-21swapper/008:01:350
18229997931,45cyclictest22236-21df_inode08:50:310
18233997829,21cyclictest7950irq/56-dwc_otg_09:45:432
18229997827,40cyclictest0-21swapper/008:33:560
18229997822,33cyclictest20107-21ssh11:37:090
1823399774,45cyclictest0-21swapper/212:35:182
18233997730,28cyclictest7950irq/56-dwc_otg_10:15:312
18229997725,26cyclictest18624-21ssh10:55:490
18229997720,47cyclictest0-21swapper/011:57:350
1823399768,37cyclictest2611-21grep12:35:172
18229997633,28cyclictest3517-21taskset07:58:580
18229997629,27cyclictest27159-21sshd09:03:400
18229997626,30cyclictest0-21swapper/007:29:440
18229997622,40cyclictest0-21swapper/009:08:590
64842750,2sleep20-21swapper/212:00:292
18233997511,44cyclictest7850irq/56-dwc_otg10:55:412
18229997527,33cyclictest0-21swapper/008:50:090
200432740,5sleep11823299cyclictest11:37:011
18229997324,32cyclictest1476-21chrt07:50:590
18229997324,32cyclictest1476-21chrt07:50:590
18229997321,33cyclictest21333-21seq07:16:450
132152730,2sleep20-21swapper/208:25:352
132152730,2sleep20-21swapper/208:25:352
131682730,4sleep337-21rcuc/312:08:243
18229997227,28cyclictest22709-21ssh12:20:160
18229997223,39cyclictest17538-21ssh11:34:590
18229997223,37cyclictest0-21swapper/008:40:430
18234996914,26cyclictest8050irq/56-dwc_otg_10:55:403
37392680,3sleep337-21rcuc/308:00:153
18229996825,34cyclictest22344-21irqstats07:20:360
18229996825,34cyclictest22344-21irqstats07:20:360
18229996821,44cyclictest195262sleep007:11:420
18232996620,27cyclictest0-21swapper/110:55:411
18229996218,27cyclictest0-21swapper/008:05:320
1823499597,39cyclictest8050irq/56-dwc_otg_07:25:463
18234995913,23cyclictest7950irq/56-dwc_otg_12:35:163
18234995913,23cyclictest7950irq/56-dwc_otg_12:35:163
1823299596,31cyclictest7950irq/56-dwc_otg_10:15:311
18234995813,14cyclictest7850irq/56-dwc_otg10:15:333
1823299573,18cyclictest29035-21ssh10:30:261
1823299573,18cyclictest29035-21ssh10:30:261
1823499569,21cyclictest0-21swapper/311:35:453
1823399568,29cyclictest8050irq/56-dwc_otg_11:35:452
1823299561,12cyclictest0-21swapper/110:10:231
1823299538,33cyclictest0-21swapper/110:05:281
1823299532,10cyclictest0-21swapper/112:00:271
34422510,5sleep23441-21ssh12:35:362
1823399518,29cyclictest0-21swapper/207:30:312
1823399518,26cyclictest0-21swapper/209:00:302
1823399515,8cyclictest8050irq/56-dwc_otg_09:30:272
1823499502,3cyclictest0-21swapper/309:45:163
1823399503,15cyclictest26982-21ssh12:25:302
1823299508,29cyclictest0-21swapper/109:10:301
1823499492,28cyclictest0-21swapper/312:35:183
1823299494,25cyclictest0-21swapper/112:35:191
18232994912,25cyclictest0-21swapper/108:40:201
9772481,5sleep11073-21cat09:55:351
1823499488,24cyclictest0-21swapper/307:30:313
18232994710,25cyclictest0-21swapper/108:45:291
178082473,36sleep00-21swapper/007:08:150
18234994620,5cyclictest29088-21cron09:10:153
18233994620,6cyclictest7850irq/56-dwc_otg09:57:232
18233994618,13cyclictest301ktimersoftd/210:10:392
1823299462,3cyclictest0-21swapper/108:50:271
18233994514,22cyclictest5097-21wget09:20:302
1823499449,24cyclictest581-21ssh09:55:293
1823499447,22cyclictest0-21swapper/308:40:213
18233994418,11cyclictest8050irq/56-dwc_otg_10:26:032
18232994416,7cyclictest1062550kworker/u9:209:35:431
5132430,2sleep30-21swapper/311:52:583
1823499438,26cyclictest29708-21ls09:50:393
18234994318,5cyclictest37-21rcuc/309:19:403
1823399437,22cyclictest0-21swapper/208:00:282
1823499427,23cyclictest0-21swapper/308:00:283
1823499427,21cyclictest0-21swapper/309:00:303
18234994222,9cyclictest39-21ksoftirqd/311:40:283
1823499421,34cyclictest10184-21ls09:25:473
18234994210,23cyclictest0-21swapper/309:30:323
1823399429,15cyclictest301ktimersoftd/211:40:272
1823399426,22cyclictest14541-21ssh12:10:282
18233994210,10cyclictest29-21rcuc/208:55:322
18233994210,10cyclictest29-21rcuc/208:55:322
1823299428,22cyclictest0-21swapper/107:35:291
1823299422,8cyclictest0-21swapper/108:25:281
1823299422,8cyclictest0-21swapper/108:25:281
18234994131,6cyclictest25521-21ssh11:05:263
1823399415,24cyclictest13819-21ssh11:30:212
1823299419,20cyclictest0-21swapper/109:40:291
1823299419,18cyclictest0-21swapper/108:55:221
1823299419,18cyclictest0-21swapper/108:55:221
1823299415,23cyclictest8050irq/56-dwc_otg_09:15:441
299362401,33sleep130064-21ssh11:10:331
18234994029,4cyclictest8050irq/56-dwc_otg_09:35:463
1823499402,25cyclictest4887-21fschecks_time10:00:323
1823399409,6cyclictest31-21ksoftirqd/208:20:372
18233994019,10cyclictest8050irq/56-dwc_otg_09:18:352
1823399401,25cyclictest0-21swapper/208:40:212
18232994016,14cyclictest25650-21/usr/sbin/munin09:45:361
18232994015,9cyclictest22157-21ssh11:40:271
1823499397,17cyclictest29452-21/usr/sbin/munin09:10:343
1823499396,23cyclictest25042-21ssh09:45:283
1823499393,23cyclictest7950irq/56-dwc_otg_11:19:153
1823399398,19cyclictest0-21swapper/209:50:302
1823399393,31cyclictest21392-21ssh11:00:252
1823299394,22cyclictest8050irq/56-dwc_otg_10:00:441
1823299393,7cyclictest8050irq/56-dwc_otg_09:20:341
18232993920,7cyclictest6672-21timerwakeupswit11:20:421
278122380,5sleep11823299cyclictest10:28:371
1823499383,16cyclictest39-21ksoftirqd/310:10:173
18234993826,4cyclictest8050irq/56-dwc_otg_10:20:313
18234993810,21cyclictest0-21swapper/310:08:043
1823399389,10cyclictest301ktimersoftd/212:20:192
1823399388,13cyclictest7950irq/56-dwc_otg_09:25:352
1823399387,6cyclictest0-21swapper/211:55:292
1823399387,22cyclictest0-21swapper/211:20:282
1823399387,19cyclictest29-21rcuc/207:40:422
1823399386,22cyclictest0-21swapper/209:13:382
18233993817,5cyclictest12486-21ssh11:28:312
18233993811,11cyclictest301ktimersoftd/208:50:282
1823299382,9cyclictest25587-21wget07:30:321
1823299382,10cyclictest0-21swapper/107:55:301
1823299381,27cyclictest12698-21latency_hist09:30:161
1823399379,16cyclictest8050irq/56-dwc_otg_10:00:442
1823399378,11cyclictest29-21rcuc/207:30:052
1823399375,12cyclictest301ktimersoftd/209:35:472
18233993717,11cyclictest23671-21ssh09:43:192
18233993711,9cyclictest31-21ksoftirqd/208:20:162
1823399371,16cyclictest8050irq/56-dwc_otg_10:45:302
1823299377,5cyclictest7950irq/56-dwc_otg_07:25:481
1823299374,20cyclictest8050irq/56-dwc_otg_09:30:451
18232993715,6cyclictest26692-1kworker/u9:109:50:271
18232993711,16cyclictest23435-21ssh10:23:081
1823499366,15cyclictest0-21swapper/312:10:283
1823499365,5cyclictest8050irq/56-dwc_otg_08:25:453
1823499365,5cyclictest8050irq/56-dwc_otg_08:25:453
1823499365,19cyclictest39-21ksoftirqd/308:55:303
1823499365,19cyclictest39-21ksoftirqd/308:55:303
1823499363,10cyclictest8050irq/56-dwc_otg_09:20:383
1823499362,4cyclictest8050irq/56-dwc_otg_10:25:263
1823399364,15cyclictest301ktimersoftd/211:05:432
1823399361,4cyclictest0-21swapper/208:30:312
18233993611,5cyclictest7850irq/56-dwc_otg12:15:282
18233993610,11cyclictest301ktimersoftd/210:08:192
1823299364,8cyclictest0-21swapper/112:20:151
1823299362,9cyclictest11422-21fschecks_time08:20:341
1823299362,3cyclictest7950irq/56-dwc_otg_08:15:291
18232993621,4cyclictest757150wget08:10:321
18232993614,13cyclictest0-21swapper/112:20:381
18232993611,3cyclictest0-21swapper/111:16:221
18232993610,7cyclictest0-21swapper/112:11:091
1823499356,5cyclictest8050irq/56-dwc_otg_07:40:153
1823499356,21cyclictest0-21swapper/312:00:273
1823499356,15cyclictest0-21swapper/311:20:283
1823499356,10cyclictest39-21ksoftirqd/311:45:433
1823499355,16cyclictest0-21swapper/308:35:303
1823499354,22cyclictest0-21swapper/310:41:013
1823499354,21cyclictest39-21ksoftirqd/312:25:453
18234993518,5cyclictest23717-21apt-config08:55:163
1823499351,23cyclictest8050irq/56-dwc_otg_08:05:343
18234993511,15cyclictest39-21ksoftirqd/310:54:573
1823399356,17cyclictest301ktimersoftd/208:15:162
1823399356,15cyclictest29-21rcuc/211:19:142
1823399356,13cyclictest31-21ksoftirqd/211:49:002
1823399355,20cyclictest30455-21ssh11:50:282
1823399355,17cyclictest301ktimersoftd/212:09:012
1823399355,17cyclictest301ktimersoftd/210:41:552
1823399354,13cyclictest301ktimersoftd/207:14:202
1823399351,20cyclictest31-21ksoftirqd/211:13:252
1823299357,17cyclictest0-21swapper/111:30:221
1823299355,19cyclictest5979-21fschecks_time08:05:351
1823299353,20cyclictest7850irq/56-dwc_otg11:06:261
1823299352,7cyclictest0-21swapper/107:10:271
1823299352,21cyclictest23-21ksoftirqd/111:55:161
18232993516,4cyclictest4540-21sh10:40:151
18232993514,8cyclictest0-21swapper/109:05:281
18232993514,5cyclictest23445-21idleruntime-cro07:25:161
18232993514,5cyclictest23445-21idleruntime-cro07:25:161
1823299351,23cyclictest26310-21ssh11:45:281
1823299351,18cyclictest7850irq/56-dwc_otg10:45:161
1823299351,16cyclictest0-21swapper/109:00:411
1823499346,14cyclictest381ktimersoftd/312:20:163
1823499345,18cyclictest7770-21sort08:10:353
18234993414,5cyclictest1114-21fschecks_time10:35:323
18234993413,10cyclictest381ktimersoftd/307:45:413
18234993411,13cyclictest37-21rcuc/311:10:433
1823399346,7cyclictest29-21rcuc/210:33:462
1823399346,7cyclictest29-21rcuc/210:33:462
1823399341,8cyclictest31-21ksoftirqd/210:50:312
18233993411,10cyclictest7850irq/56-dwc_otg10:23:422
18233993410,10cyclictest7850irq/56-dwc_otg07:45:392
18232993416,10cyclictest22023-21ssh11:00:351
1823299341,3cyclictest0-21swapper/108:00:331
18232993412,3cyclictest7950irq/56-dwc_otg_07:40:291
1823299341,23cyclictest31500-21missed_timers07:45:391
1823299341,21cyclictest14288-21ssh12:10:161
1823399338,12cyclictest7950irq/56-dwc_otg_10:35:262
1823399331,8cyclictest6850-21unixbench_singl08:05:482
18233993311,4cyclictest29-21rcuc/207:20:272
18233993311,4cyclictest29-21rcuc/207:20:272
1823299336,6cyclictest7850irq/56-dwc_otg12:30:151
1823299336,15cyclictest0-21swapper/111:25:281
1823299331,11cyclictest750-21snmpd11:58:391
176982333,22sleep10-21swapper/107:06:551
18234993217,9cyclictest29093-21kworker/u8:111:57:493
1823499321,17cyclictest39-21ksoftirqd/308:20:163
1823499321,17cyclictest1983-21ls07:55:163
1823499321,17cyclictest1983-21ls07:55:163
1823499321,17cyclictest0-21swapper/311:02:403
1823399329,5cyclictest29246-21latency_hist09:10:162
1823399322,10cyclictest0-21swapper/208:39:142
1823299321,10cyclictest14180-21ssh10:50:431
1823499318,4cyclictest39-21ksoftirqd/310:45:343
1823499317,16cyclictest39-21ksoftirqd/307:11:393
1823499313,13cyclictest37-21rcuc/311:26:553
1823399316,13cyclictest301ktimersoftd/208:50:162
1823299316,5cyclictest20179-21wget07:15:311
1823299311,16cyclictest0-21swapper/108:32:021
1823499304,5cyclictest37-21rcuc/308:20:373
1823499303,12cyclictest39-21ksoftirqd/312:20:163
1823499301,9cyclictest15020-21grep08:30:353
18234993012,9cyclictest20442-21diskstats08:45:323
1823399308,11cyclictest7950irq/56-dwc_otg_07:55:432
1823399301,18cyclictest0-21swapper/207:35:222
1823299306,14cyclictest0-21swapper/107:50:291
1823299306,14cyclictest0-21swapper/107:50:291
1823299305,4cyclictest516-21in:imuxsock08:38:581
18234992911,6cyclictest7950irq/56-dwc_otg_07:40:363
1823499291,15cyclictest8050irq/56-dwc_otg_07:15:433
1823399292,6cyclictest8050irq/56-dwc_otg_07:16:462
1823299291,9cyclictest9015-21ssh10:45:271
1823499285,7cyclictest8050irq/56-dwc_otg_07:20:453
1823499285,7cyclictest8050irq/56-dwc_otg_07:20:453
1823399282,8cyclictest0-21swapper/207:50:272
1823399282,8cyclictest0-21swapper/207:50:272
165052283,16sleep216617-21df07:05:302
176852264,14sleep30-21swapper/307:06:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional