You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 14:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat May 30, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2319222950,3sleep00-21swapper/019:10:060
23280991076,79cyclictest19223-21unixbench-2d20:25:330
232809910346,13cyclictest0-21swapper/021:30:160
69672980,2sleep00-21swapper/021:54:260
296502940,2sleep30-21swapper/319:25:313
2328099931,78cyclictest17925-21ssh21:28:310
2328099922,73cyclictest7404-21fschecks_time19:55:230
194412920,2sleep00-21swapper/000:05:360
2328099882,3cyclictest0-21swapper/022:15:240
2328099881,7cyclictest18898-21users23:25:350
2328099881,5cyclictest27839-21wget21:40:200
286552860,4sleep328656-21latency_hist19:25:063
2328099862,71cyclictest0-21swapper/000:25:230
2328099862,71cyclictest0-21swapper/000:25:230
2328099861,3cyclictest0-21swapper/023:20:150
2328099852,70cyclictest0-21swapper/022:40:300
2328099852,67cyclictest31383-21wc19:30:310
2328099851,5cyclictest28925-21munin-run23:00:060
2328099841,77cyclictest0-21swapper/021:20:170
2328099841,4cyclictest30472-21latency_hist00:20:060
2328099821,4cyclictest0-21swapper/019:25:210
101012810,1sleep10-21swapper/100:34:471
2328099806,65cyclictest0-21swapper/022:48:550
2328099801,65cyclictest9-21ksoftirqd/000:15:050
2328099801,5cyclictest2915-21unixbench_multi23:45:330
202472800,2sleep10-21swapper/123:28:001
23280997815,46cyclictest23441-21/usr/sbin/munin21:35:250
231052780,6sleep28050irq/56-dwc_otg_22:13:292
23283997710,13cyclictest7850irq/56-dwc_otg21:45:173
23283997710,13cyclictest7850irq/56-dwc_otg21:45:173
2328099771,6cyclictest5784-21chrt21:14:440
2328099761,5cyclictest6703-21ssh22:32:290
2328099761,58cyclictest14110-21ssh00:39:340
2328099752,8cyclictest0-21swapper/000:22:520
2328099752,63cyclictest0-21swapper/021:56:010
2328099752,56cyclictest0-21swapper/020:50:290
2328099751,5cyclictest0-21swapper/023:05:500
2328099751,50cyclictest0-21swapper/020:40:230
2328099751,50cyclictest0-21swapper/020:40:230
2328099751,4cyclictest31540-21rm22:23:440
2328099751,4cyclictest24044-21latency_hist22:15:060
2328099746,58cyclictest10944-21sh22:37:410
2328099742,3cyclictest0-21swapper/023:39:520
2328099741,6cyclictest30278-21timerandwakeup23:00:320
2328099741,6cyclictest30278-21timerandwakeup23:00:320
2328099741,4cyclictest31935-21ssh21:45:200
2328099741,4cyclictest31935-21ssh21:45:200
2328099741,3cyclictest0-21swapper/023:42:420
181382740,1sleep30-21swapper/322:45:433
2328099732,63cyclictest9452-21ssh21:18:390
2328099732,63cyclictest9452-21ssh21:18:390
2328099731,5cyclictest11865-21cut22:00:190
2328099731,59cyclictest22908-21sh22:51:430
2328099722,54cyclictest0-21swapper/023:15:210
2328099721,61cyclictest8013-21ssh00:30:570
2328099721,5cyclictest0-21swapper/020:55:340
131612720,1sleep20-21swapper/222:00:362
2328099711,5cyclictest0-21swapper/023:30:290
2328099711,4cyclictest17150-21ps20:20:300
2328099711,4cyclictest16154-21runrttasks20:20:060
2328099711,4cyclictest14029-21ssh00:00:100
2328099711,4cyclictest14029-21ssh00:00:100
2328099702,54cyclictest0-21swapper/019:17:070
2328099701,5cyclictest6773-21switchtime23:50:310
2328099701,5cyclictest12307-21chrt20:09:150
2328099701,54cyclictest8542-21seq23:14:180
2328099701,4cyclictest21406-21chrt20:33:460
2328099701,4cyclictest0-21swapper/022:26:350
2328099699,35cyclictest1485750kworker/u9:122:06:070
2328099691,4cyclictest14193-21apt-get20:15:060
2328099691,4cyclictest14193-21apt-get20:15:060
2328099691,4cyclictest13590-21ssh23:59:480
2328299683,15cyclictest0-21swapper/220:45:172
2328199687,48cyclictest0-21swapper/100:25:221
2328199687,48cyclictest0-21swapper/100:25:221
23281996825,11cyclictest0-21swapper/120:45:171
2328099682,7cyclictest0-21swapper/019:41:370
2328099681,7cyclictest1734-21munin-run21:10:060
2328099671,5cyclictest23251-21chrt20:39:140
234472660,2sleep10-21swapper/122:52:411
2328099661,5cyclictest28255-21chrt19:22:170
2328099661,5cyclictest0-21swapper/021:03:460
2328099661,4cyclictest24768-21chrt19:13:560
2328099651,4cyclictest0-21swapper/019:36:020
2328099651,3cyclictest102682sleep020:01:220
277022640,2sleep30-21swapper/323:36:323
23280996410,35cyclictest25371-21/usr/sbin/munin20:45:220
2328099622,39cyclictest4358-21wc19:45:310
2328099622,39cyclictest4358-21wc19:45:310
2328099611,4cyclictest5305-21apt-get19:50:160
2328099611,4cyclictest5305-21apt-get19:50:160
23282996014,16cyclictest19164-21ssh21:30:162
2328299574,10cyclictest0-21swapper/219:20:152
23282995618,23cyclictest11996-21sh21:20:392
23282995411,24cyclictest32299-21/usr/sbin/munin21:45:242
23282995411,24cyclictest32299-21/usr/sbin/munin21:45:242
23281995412,9cyclictest22415-21sh00:10:191
23282995311,21cyclictest4590-21ssh23:48:282
2328199534,9cyclictest0-21swapper/123:50:181
23282995111,27cyclictest8050irq/56-dwc_otg_21:50:192
2328199517,30cyclictest19179-21ssh21:30:171
229002513,6sleep10-21swapper/119:08:381
2328399508,11cyclictest8050irq/56-dwc_otg_00:30:163
2328299509,26cyclictest28341-21awk20:50:332
2328299497,26cyclictest4342-21ssh22:30:172
23282994910,25cyclictest14429-21cat20:15:072
23282994910,25cyclictest14429-21cat20:15:072
23281994931,12cyclictest221ktimersoftd/121:20:181
2328299489,22cyclictest31955-21rm23:41:472
2328299472,10cyclictest0-21swapper/200:20:172
2328199476,28cyclictest11847-21ssh22:00:191
2328199475,23cyclictest0-21swapper/120:50:161
2328299464,23cyclictest8050irq/56-dwc_otg_23:57:052
2328299462,24cyclictest7950irq/56-dwc_otg_22:15:302
23282994617,5cyclictest301ktimersoftd/219:55:282
2328299461,16cyclictest16693-21ssh22:05:272
23281994615,23cyclictest9020-21runrttasks20:00:151
2328399458,26cyclictest0-21swapper/320:40:193
2328399458,26cyclictest0-21swapper/320:40:193
2328399457,26cyclictest0-21swapper/320:05:203
2328399457,25cyclictest0-21swapper/319:45:183
2328399457,25cyclictest0-21swapper/319:45:183
2328299457,24cyclictest28508-21seq22:20:192
2328299454,19cyclictest7850irq/56-dwc_otg23:20:192
23282994513,17cyclictest7850irq/56-dwc_otg23:00:342
23282994513,17cyclictest7850irq/56-dwc_otg23:00:342
23282994513,10cyclictest8050irq/56-dwc_otg_00:15:172
2328199457,25cyclictest0-21swapper/123:10:181
10592450,2sleep12328199cyclictest23:05:211
2328399448,25cyclictest32633-21ssh22:25:173
2328299447,24cyclictest0-21swapper/223:10:182
2328299447,23cyclictest0-21swapper/219:25:182
23282994418,17cyclictest8050irq/56-dwc_otg_22:40:332
2328199447,25cyclictest0-21swapper/123:30:181
2328199447,23cyclictest0-21swapper/119:55:191
2328199443,18cyclictest221ktimersoftd/122:15:231
23281994426,9cyclictest7950irq/56-dwc_otg_22:57:181
2328399439,21cyclictest0-21swapper/300:00:233
2328399439,21cyclictest0-21swapper/300:00:233
2328299438,21cyclictest14489-21cp20:15:072
2328299437,23cyclictest0-21swapper/222:45:182
2328299437,23cyclictest0-21swapper/220:30:192
2328299437,21cyclictest2856-21diskmemload00:25:272
2328299437,21cyclictest2856-21diskmemload00:25:272
23282994311,21cyclictest25660-21ssh00:14:082
2328199437,25cyclictest0-21swapper/122:05:111
2328199437,22cyclictest0-21swapper/120:30:191
2328399427,22cyclictest15993-21ssh22:05:123
2328399422,8cyclictest7950irq/56-dwc_otg_00:05:183
2328299429,19cyclictest22175-1kworker/u9:221:40:262
2328299429,10cyclictest0-21swapper/221:15:102
2328299429,10cyclictest0-21swapper/221:15:102
2328299427,12cyclictest16339-21ssh21:25:462
2328299426,24cyclictest0-21swapper/220:20:192
2328299425,23cyclictest7850irq/56-dwc_otg23:37:272
2328299423,13cyclictest31708-21who19:30:342
23282994211,11cyclictest23480-21ssh23:31:212
2328199427,22cyclictest0-21swapper/121:45:191
2328199427,22cyclictest0-21swapper/121:45:191
2328199427,22cyclictest0-21swapper/100:05:211
2328199424,9cyclictest7850irq/56-dwc_otg23:40:181
23281994217,5cyclictest9839-21sh22:35:391
2328399418,23cyclictest0-21swapper/320:15:113
2328399417,25cyclictest0-21swapper/319:50:193
2328399417,25cyclictest0-21swapper/319:50:193
2328399417,24cyclictest0-21swapper/323:20:183
2328399412,7cyclictest1917-21latency_hist21:10:073
2328299418,23cyclictest0-21swapper/221:59:102
2328299417,21cyclictest0-21swapper/223:15:172
2328299417,21cyclictest0-21swapper/219:40:182
2328299417,11cyclictest8486-21ssh22:35:182
2328299416,20cyclictest27245-21ssh21:39:542
2328199417,19cyclictest21-21rcuc/120:20:061
2328399407,23cyclictest0-21swapper/322:35:203
2328399407,23cyclictest0-21swapper/320:00:173
2328399406,23cyclictest0-21swapper/321:50:193
23283994027,8cyclictest1721-21diskmemload21:20:183
23283994021,7cyclictest24793-21idleruntime-cro22:55:063
2328299409,13cyclictest19083-21/usr/sbin/munin00:05:322
2328299406,12cyclictest1033-21apt-get23:05:192
23282994021,8cyclictest0-21swapper/221:12:262
23282994018,9cyclictest8245-21ssh00:31:232
2328199407,23cyclictest0-21swapper/123:00:181
2328199407,23cyclictest0-21swapper/123:00:181
2328199406,15cyclictest0-21swapper/121:42:291
2328199402,15cyclictest221ktimersoftd/121:15:061
23281994021,4cyclictest21-21rcuc/119:50:061
23281994021,4cyclictest21-21rcuc/119:50:061
23281994020,8cyclictest221ktimersoftd/123:45:341
23281994015,6cyclictest23-21ksoftirqd/123:40:071
23281994015,6cyclictest17074-21sh21:27:031
2328399395,12cyclictest0-21swapper/319:40:163
2328399392,9cyclictest7659-21apt-get21:55:163
23282993924,8cyclictest7950irq/56-dwc_otg_00:02:452
23282993924,8cyclictest7950irq/56-dwc_otg_00:02:452
23281993923,5cyclictest31556-21/usr/sbin/munin00:20:301
23281993919,5cyclictest23-21ksoftirqd/123:20:061
2328199391,8cyclictest791750ssh21:55:211
2328399385,3cyclictest7850irq/56-dwc_otg23:05:203
2328399385,15cyclictest7950irq/56-dwc_otg_21:40:073
2328399382,17cyclictest8050irq/56-dwc_otg_20:40:063
2328299386,3cyclictest0-21swapper/222:50:072
2328299385,21cyclictest1165-21snmpd22:26:342
23282993818,11cyclictest7950irq/56-dwc_otg_21:09:212
23282993817,5cyclictest7850irq/56-dwc_otg20:45:052
23282993817,5cyclictest7850irq/56-dwc_otg20:45:052
23282993814,18cyclictest301ktimersoftd/222:57:412
23282993813,14cyclictest301ktimersoftd/223:25:382
2328299381,23cyclictest11021-21ls20:05:222
23282993811,15cyclictest8050irq/56-dwc_otg_23:51:432
2328199388,6cyclictest7850irq/56-dwc_otg21:35:341
23281993810,5cyclictest23-21ksoftirqd/121:53:581
2328399377,21cyclictest0-21swapper/319:30:183
23283993717,5cyclictest589450wget23:50:193
23282993716,6cyclictest301ktimersoftd/200:37:032
23282993711,18cyclictest5291-21/usr/sbin/munin19:50:202
23282993711,18cyclictest5291-21/usr/sbin/munin19:50:202
2328199377,15cyclictest0-21swapper/122:30:171
2328199374,13cyclictest0-21swapper/119:25:171
23281993716,18cyclictest20666-21latency_hist22:50:071
23281993715,4cyclictest3268150wget21:05:201
23281993713,10cyclictest221ktimersoftd/121:17:501
23281993713,10cyclictest221ktimersoftd/121:17:501
23283993616,5cyclictest26191-21basename00:15:053
2328199366,16cyclictest0-21swapper/122:20:191
2328199363,20cyclictest7850irq/56-dwc_otg22:26:301
23281993618,11cyclictest7850irq/56-dwc_otg00:16:501
23281993614,4cyclictest18890-21/usr/sbin/munin20:25:301
23281993613,5cyclictest0-21swapper/122:10:191
23283993524,4cyclictest8050irq/56-dwc_otg_21:18:333
23283993524,4cyclictest8050irq/56-dwc_otg_21:18:333
2328299354,19cyclictest7950irq/56-dwc_otg_19:35:322
2328299353,23cyclictest7950irq/56-dwc_otg_20:35:272
2328199353,8cyclictest0-21swapper/119:30:161
23281993515,10cyclictest221ktimersoftd/122:40:221
23281993513,4cyclictest23388-21aten_rbpower_po19:10:191
2328199351,19cyclictest0-21swapper/100:03:281
2328199351,19cyclictest0-21swapper/100:03:281
2328199351,15cyclictest23-21ksoftirqd/120:20:311
2328399347,14cyclictest8050irq/56-dwc_otg_23:45:193
2328399344,7cyclictest7850irq/56-dwc_otg22:20:353
2328399343,15cyclictest28348-21latency_hist22:20:073
23283993416,5cyclictest8646-21sh23:14:293
23283993414,5cyclictest7982-21sh22:34:493
2328399341,25cyclictest0-21swapper/322:58:083
2328299347,22cyclictest0-21swapper/220:00:122
2328299344,20cyclictest7950irq/56-dwc_otg_21:00:352
23282993416,8cyclictest7950irq/56-dwc_otg_19:14:022
23282993414,5cyclictest25523-21if_eth019:15:232
2328199343,16cyclictest21-21rcuc/100:37:021
23281993410,10cyclictest221ktimersoftd/123:20:151
2328399335,16cyclictest7850irq/56-dwc_otg00:20:063
2328399335,16cyclictest0-21swapper/321:10:263
23283993317,14cyclictest8050irq/56-dwc_otg_22:40:283
23283993315,12cyclictest1133-21runrttasks00:27:533
23283993315,12cyclictest1133-21runrttasks00:27:533
23283993313,6cyclictest15746-21runrttasks22:05:063
23283993313,4cyclictest29754-21sh23:00:253
23283993313,4cyclictest29754-21sh23:00:253
2328399331,22cyclictest19743-21latency21:30:243
2328399331,18cyclictest30657-21ssh00:20:163
2328199337,15cyclictest221ktimersoftd/100:00:061
2328199332,19cyclictest7850irq/56-dwc_otg19:50:171
2328199332,19cyclictest7850irq/56-dwc_otg19:50:171
2328199331,4cyclictest7850irq/56-dwc_otg19:40:261
23281993312,12cyclictest221ktimersoftd/120:42:181
23281993312,12cyclictest221ktimersoftd/120:42:181
214512336,14sleep20-21swapper/219:05:152
23283993220,10cyclictest8050irq/56-dwc_otg_21:25:193
23283993215,4cyclictest14212-21sh00:39:453
2328399321,21cyclictest28292-21latency21:40:263
2328399321,18cyclictest8050irq/56-dwc_otg_20:15:063
2328399321,18cyclictest8050irq/56-dwc_otg_20:15:063
23283993210,7cyclictest0-21swapper/320:28:003
2328299321,12cyclictest30432-21taskset20:58:462
2328199329,11cyclictest221ktimersoftd/120:55:341
2328199327,11cyclictest7850irq/56-dwc_otg19:20:191
23281993211,4cyclictest23559-21date20:40:071
2328399317,15cyclictest0-21swapper/323:55:193
2328399316,13cyclictest8050irq/56-dwc_otg_23:25:203
2328399313,20cyclictest21085-21unixbench_multi20:30:343
2328399311,6cyclictest0-21swapper/319:55:173
2328199318,13cyclictest221ktimersoftd/121:01:111
2328399308,11cyclictest0-21swapper/323:43:063
2328399307,12cyclictest8050irq/56-dwc_otg_23:16:293
2328399303,16cyclictest0-21swapper/320:50:263
2328399303,12cyclictest7950irq/56-dwc_otg_20:20:263
2328399302,6cyclictest8050irq/56-dwc_otg_21:00:193
2328299302,4cyclictest8050irq/56-dwc_otg_20:25:162
2328299301,14cyclictest7950irq/56-dwc_otg_19:45:162
2328299301,14cyclictest7950irq/56-dwc_otg_19:45:162
2328199305,5cyclictest8050irq/56-dwc_otg_20:10:191
2328199305,5cyclictest8050irq/56-dwc_otg_20:10:191
2328199302,4cyclictest0-21swapper/119:15:331
23281993017,6cyclictest21-21rcuc/119:39:021
2328399294,5cyclictest8050irq/56-dwc_otg_19:10:233
2328399292,7cyclictest22563-21ssh23:30:303
23283992921,4cyclictest30551-21latency_hist21:00:053
2328399291,8cyclictest25374-21fschecks_count19:15:213
23283992911,5cyclictest24007-21latency_hist22:15:063
2328199296,6cyclictest7850irq/56-dwc_otg20:05:211
2328399287,11cyclictest7950irq/56-dwc_otg_19:35:343
23283992824,2cyclictest0-21swapper/320:50:063
227412263,15sleep30-21swapper/319:06:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional