You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 13:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Jul 18, 2026 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2216222710,1sleep30-21swapper/319:05:423
232809917218,122cyclictest0-21swapper/023:10:350
232809914676,45cyclictest7950irq/56-dwc_otg_22:45:330
232829912614,22cyclictest6435-21ssh21:15:362
232819911715,43cyclictest7850irq/56-dwc_otg23:10:351
23280991114,16cyclictest19838-21grep21:30:490
23280991095,73cyclictest0-21swapper/022:35:380
232809910865,19cyclictest101ktimersoftd/021:25:350
232809910857,32cyclictest0-21swapper/023:15:460
232849910717,68cyclictest0-21swapper/321:25:343
232829910415,18cyclictest7850irq/56-dwc_otg22:05:232
232829910414,17cyclictest7950irq/56-dwc_otg_23:20:372
232829910414,17cyclictest7950irq/56-dwc_otg_23:20:372
232819910331,45cyclictest9237-21ssh23:15:451
23280991023,80cyclictest0-21swapper/022:16:260
23280991023,64cyclictest0-21swapper/021:15:360
232809910210,71cyclictest9-21ksoftirqd/023:20:360
232809910210,71cyclictest9-21ksoftirqd/023:20:360
232819910136,24cyclictest14685-21cat21:25:371
232809910155,23cyclictest0-21swapper/022:55:400
2328299973,11cyclictest29807-21ntp_states23:40:432
23281999626,23cyclictest8169-21ssh22:35:381
23281999540,39cyclictest1594-21diskmemload21:30:491
23280999520,51cyclictest0-21swapper/022:50:400
23280999520,51cyclictest0-21swapper/022:50:400
23280999464,17cyclictest14688-21kworker/0:222:05:240
23284999323,44cyclictest8050irq/56-dwc_otg_23:40:423
2328299933,14cyclictest19789-21ssh21:30:492
2328299913,13cyclictest14670-21ls21:25:362
2328099917,44cyclictest101ktimersoftd/000:05:430
23280999140,30cyclictest12130-21fschecks_time22:40:380
2328299902,14cyclictest7950irq/56-dwc_otg_22:55:402
23281999031,29cyclictest1289-21sed23:05:481
2328099904,41cyclictest0-21swapper/000:10:460
2328299893,12cyclictest18165-21ntp_states00:05:442
23282998821,9cyclictest4361-21aten_rbpower_vo23:10:372
23281998828,28cyclictest12811-21grep23:20:381
23281998828,28cyclictest12811-21grep23:20:381
23282998713,9cyclictest7950irq/56-dwc_otg_00:10:512
23281998629,22cyclictest24672-21ssh22:55:411
2328299856,17cyclictest8050irq/56-dwc_otg_23:15:462
23281998527,20cyclictest7950irq/56-dwc_otg_00:05:411
2328099835,67cyclictest0-21swapper/021:40:300
23280998324,29cyclictest0-21swapper/023:30:310
225072830,2sleep10-21swapper/122:52:571
225072830,2sleep10-21swapper/122:52:571
14902830,3sleep337-21rcuc/323:45:423
23284998223,38cyclictest7950irq/56-dwc_otg_22:40:373
23284998221,22cyclictest24638-21ssh22:55:403
23281998224,33cyclictest0-21swapper/100:10:511
2328099813,45cyclictest0-21swapper/021:00:310
163092810,7sleep121-21rcuc/122:45:401
23284998029,24cyclictest8173-21ssh22:35:393
23284998017,32cyclictest8050irq/56-dwc_otg_00:05:423
2328099794,45cyclictest0-21swapper/023:05:490
23281997826,24cyclictest12038-21diskstats22:40:361
23281997824,21cyclictest7850irq/56-dwc_otg22:16:261
2328099783,49cyclictest0-21swapper/023:40:430
23284997731,24cyclictest727-21runrttasks22:05:233
2328499772,45cyclictest16141-21ls22:45:353
23284997711,36cyclictest7950irq/56-dwc_otg_23:05:483
2328499767,35cyclictest20749-21ssh23:30:313
2328499765,45cyclictest4272-21ssh23:10:353
23284997639,21cyclictest8317-21/usr/sbin/munin23:15:463
2328099751,53cyclictest32230-21latency_hist19:35:200
23281997418,21cyclictest7850irq/56-dwc_otg23:40:411
2328099747,29cyclictest0-21swapper/019:40:320
2328099746,41cyclictest0-21swapper/000:30:320
2328299733,9cyclictest20742-21apt-get23:30:322
2328299733,9cyclictest20665-21memory22:50:422
2328299733,9cyclictest20665-21memory22:50:422
23280997325,9cyclictest0-21swapper/020:00:290
2328099731,59cyclictest0-21swapper/019:25:460
88772720,2sleep10-21swapper/121:57:041
23284997218,31cyclictest0-21swapper/323:20:353
23284997218,31cyclictest0-21swapper/323:20:353
2328299725,44cyclictest7850irq/56-dwc_otg22:45:332
111542720,2sleep20-21swapper/220:05:382
23281997021,28cyclictest7850irq/56-dwc_otg21:15:351
23280997045,5cyclictest8050irq/56-dwc_otg_00:15:310
23280997020,8cyclictest0-21swapper/019:20:300
23284996940,23cyclictest20549-21irqstats22:50:393
23284996940,23cyclictest20549-21irqstats22:50:393
23281996922,24cyclictest7950irq/56-dwc_otg_22:05:211
23281996820,26cyclictest20761-21ssh23:30:331
2328099681,35cyclictest0-21swapper/019:55:350
2328099681,35cyclictest0-21swapper/019:55:350
23284996717,20cyclictest7950irq/56-dwc_otg_22:16:263
2328099675,9cyclictest61-21kswapd021:50:320
2328099662,42cyclictest6455-21unixbench_singl19:50:480
2328199656,8cyclictest0-21swapper/119:55:341
2328199656,8cyclictest0-21swapper/119:55:341
2328099652,30cyclictest0-21swapper/020:55:320
2328099651,49cyclictest11139-21aten_rbpower_vo22:00:340
23284996432,26cyclictest19839-21grep21:30:493
2328299644,32cyclictest0-21swapper/223:05:492
17112640,1sleep10-21swapper/122:27:401
23284996234,22cyclictest22580-21ssh00:10:473
23284996133,22cyclictest6423-21ssh21:15:363
2328299617,15cyclictest0-21swapper/222:40:372
2328099615,34cyclictest0-21swapper/020:25:490
2328099615,34cyclictest0-21swapper/020:25:490
2328099612,40cyclictest0-21swapper/000:35:370
2328099611,43cyclictest3019-21timerandwakeup21:10:470
23282996029,9cyclictest7950irq/56-dwc_otg_21:00:312
2328199607,35cyclictest0-21swapper/100:20:341
23281996022,19cyclictest15721-21ssh00:03:061
2328099609,35cyclictest0-21swapper/022:20:350
2328099609,35cyclictest0-21swapper/022:20:350
2328099601,38cyclictest16277-21grep20:20:330
23284995914,4cyclictest8050irq/56-dwc_otg_21:40:303
23281995919,19cyclictest31154-21ls19:30:411
2328099593,26cyclictest0-21swapper/023:45:300
2328099592,37cyclictest0-21swapper/020:05:470
2328099591,39cyclictest0-21swapper/023:50:460
2328099591,39cyclictest0-21swapper/023:50:460
2328099591,38cyclictest31881-21aten_rbpower_en22:25:330
23281995820,23cyclictest25261-21/usr/sbin/munin20:45:311
23281995818,20cyclictest27045-21/usr/sbin/munin20:50:461
2328099581,38cyclictest23950-21wc20:40:400
2328099581,37cyclictest25095-21/usr/sbin/munin19:15:400
2328099581,34cyclictest12905-21latency_hist00:00:190
207712580,1sleep00-21swapper/022:10:560
2328499577,31cyclictest0-21swapper/300:35:253
23284995710,26cyclictest31724-21/usr/sbin/munin19:30:503
2328299577,11cyclictest8200-21cut22:35:392
2328099572,38cyclictest0-21swapper/021:55:350
2328099571,39cyclictest12552-21aten_rbpower_cu20:10:320
2328099571,36cyclictest20146-21fschecks_time20:30:350
2328099571,35cyclictest4607-21unixbench_multi19:45:490
2328099566,32cyclictest28326-21sh23:00:260
23280995630,16cyclictest15532-21unixbench_multi20:15:480
23281995519,22cyclictest11616-21sed20:05:461
23281995517,22cyclictest7950irq/56-dwc_otg_19:25:291
2328099552,34cyclictest0-21swapper/000:20:320
23284995415,8cyclictest8919-21apt-get20:00:293
23281995422,13cyclictest4255-21ps19:45:451
23281995412,13cyclictest20667-21head22:10:481
2328099542,43cyclictest0-21swapper/021:20:330
2328499537,27cyclictest0-21swapper/319:10:323
23281995311,26cyclictest0-21swapper/119:40:321
2328099532,39cyclictest0-21swapper/021:45:390
2328099531,38cyclictest3570-21aten_rbpower_en22:30:330
23281995216,14cyclictest26510-21rm21:40:181
23281995214,21cyclictest13533-21wc20:10:461
23284995117,19cyclictest29577-21wc19:25:453
23281995124,10cyclictest0-21swapper/100:25:341
23281995120,17cyclictest5797-21rm21:54:021
23281995120,11cyclictest6713-21ssh23:51:481
23281995120,11cyclictest6713-21ssh23:51:481
23281995117,22cyclictest16593-21/usr/sbin/munin23:25:491
23281995115,16cyclictest29485-21ssh23:00:461
23281995113,23cyclictest27496-21/usr/sbin/munin21:40:401
2328099511,38cyclictest28296-21kworker/0:019:36:120
2328499507,24cyclictest0-21swapper/321:05:333
2328299502,10cyclictest7950irq/56-dwc_otg_23:55:302
23281995020,21cyclictest14572-21ssh22:04:541
23281995013,27cyclictest8643-21ssh00:33:551
2328099502,12cyclictest0-21swapper/000:25:310
2328099501,13cyclictest0-21swapper/019:10:390
2328299498,29cyclictest0-21swapper/219:45:332
2328199496,28cyclictest0-21swapper/100:35:251
23281994920,16cyclictest31097-21ssh22:24:391
23281994920,16cyclictest31097-21ssh22:24:391
23281994919,21cyclictest29224-21ssh00:19:541
23281994910,11cyclictest0-21swapper/121:20:361
2328099497,6cyclictest17073-21latency_hist00:05:200
23284994813,11cyclictest31481-21ssh23:04:013
23284994811,22cyclictest1313-21ssh21:48:253
2328299488,28cyclictest0-21swapper/219:50:332
2328299488,25cyclictest0-21swapper/223:00:262
2328299485,14cyclictest0-21swapper/222:16:252
23281994817,20cyclictest3592-21ssh23:48:391
23281994814,24cyclictest8097-1kworker/u9:223:57:111
23281994812,22cyclictest1434-21ssh21:48:381
23281994811,28cyclictest5677-21ssh21:15:031
23280994827,6cyclictest7950irq/56-dwc_otg_21:38:020
2328099482,19cyclictest0-21swapper/023:35:380
2328499476,14cyclictest3754-21df_inode22:30:353
2328299478,29cyclictest0-21swapper/219:25:312
2328199479,10cyclictest0-21swapper/122:30:341
23280994713,14cyclictest17310-21missed_timers23:25:420
227662472,36sleep00-21swapper/019:07:140
2328499467,23cyclictest0-21swapper/320:15:253
2328499466,10cyclictest7950irq/56-dwc_otg_22:22:173
2328499466,10cyclictest7950irq/56-dwc_otg_22:22:173
23284994615,15cyclictest7123-21ssh00:31:093
2328499459,25cyclictest17497-21seq20:21:383
2328499457,23cyclictest0-21swapper/300:20:343
23284994514,18cyclictest4190-21ssh00:28:213
23284994512,21cyclictest25205-21ssh21:37:553
23281994512,16cyclictest8050irq/56-dwc_otg_23:37:271
2328499448,19cyclictest8050irq/56-dwc_otg_21:55:433
2328499447,23cyclictest0-21swapper/321:00:333
2328499444,24cyclictest8050irq/56-dwc_otg_19:52:083
23284994415,21cyclictest20861-21ssh22:11:063
23284994415,19cyclictest12994-21cat00:00:203
23284994413,11cyclictest4173-21ssh21:12:243
2328199447,26cyclictest0-21swapper/119:10:321
23281994415,15cyclictest7850irq/56-dwc_otg21:00:191
2328099442,30cyclictest0-21swapper/020:45:310
2328099441,33cyclictest0-21swapper/021:07:010
2328499439,17cyclictest8326-21taskset23:54:443
2328499439,17cyclictest8326-21taskset23:54:443
23284994315,17cyclictest18269-21ssh23:26:263
23284994315,11cyclictest5810-21ssh21:54:033
23284994313,19cyclictest381ktimersoftd/320:05:293
2328199438,12cyclictest7850irq/56-dwc_otg20:21:181
2328199437,25cyclictest0-21swapper/119:20:321
2328199437,23cyclictest0-21swapper/121:05:331
23281994310,5cyclictest0-21swapper/121:00:321
2328099432,29cyclictest0-21swapper/020:35:210
2328499427,11cyclictest0-21swapper/321:20:363
2328499427,10cyclictest0-21swapper/320:37:103
2328499423,11cyclictest0-21swapper/319:49:473
2328299428,15cyclictest7950irq/56-dwc_otg_20:00:292
23281994213,16cyclictest21113-21sshd20:31:491
2328199421,22cyclictest0-21swapper/119:35:481
2328099424,24cyclictest9-21ksoftirqd/020:50:410
2328499418,12cyclictest16061-21rm00:03:433
2328499417,28cyclictest0-21swapper/319:15:353
2328499412,25cyclictest0-21swapper/320:42:323
23284994111,13cyclictest27721-21rm00:17:113
2328199417,25cyclictest0-21swapper/119:15:351
23281994110,10cyclictest7950irq/56-dwc_otg_20:00:291
2328499405,10cyclictest8050irq/56-dwc_otg_20:10:473
23284994011,15cyclictest28077-21ssh23:39:133
23282994013,6cyclictest23140-21if_err_wlan021:35:372
2328199409,11cyclictest727-21runrttasks19:52:081
23281994015,18cyclictest22946-21runrttasks20:37:061
2328499397,11cyclictest27119-21tr20:50:343
23284993915,10cyclictest1550-21ssh22:27:223
2328299392,5cyclictest0-21swapper/223:45:292
2328299391,28cyclictest30809-21cut19:30:372
2328299391,27cyclictest11380-21cut22:00:382
2328299391,20cyclictest7332-21modprobe22:35:192
23282993911,18cyclictest8704-21latency_hist20:00:202
23282993911,18cyclictest8704-21latency_hist20:00:202
2328499386,21cyclictest750-21snmpd19:44:063
23284993812,13cyclictest26911-21/usr/sbin/munin19:20:453
23282993810,6cyclictest18132-21tr20:25:342
23282993810,6cyclictest18132-21tr20:25:342
2328199389,19cyclictest0-21swapper/120:42:461
2328199388,18cyclictest8050irq/56-dwc_otg_20:29:171
2328199388,18cyclictest8050irq/56-dwc_otg_20:29:171
2328499377,27cyclictest0-21swapper/320:25:333
2328499377,27cyclictest0-21swapper/320:25:333
2328299373,24cyclictest5992-21timerwakeupswit23:50:482
2328299373,24cyclictest5992-21timerwakeupswit23:50:482
23282993728,6cyclictest27183-21fschecks_count21:40:362
23282993717,4cyclictest30951-21sh21:45:252
2328499366,10cyclictest8050irq/56-dwc_otg_20:45:363
23284993612,12cyclictest13218-21ssh22:02:273
2328199368,16cyclictest0-21swapper/120:15:251
23284993518,13cyclictest420-21/usr/sbin/munin19:35:403
23284993512,17cyclictest21346-21sshd20:34:503
23284993512,15cyclictest30088-21seq20:56:163
2328299357,16cyclictest7950irq/56-dwc_otg_23:36:262
2328299351,4cyclictest8050irq/56-dwc_otg_19:35:322
2328299351,21cyclictest14272-21gpgv20:15:202
2292923510,18sleep20-21swapper/219:09:232
41352340,2sleep2301ktimersoftd/221:12:202
2328299347,17cyclictest0-21swapper/222:20:322
2328299347,17cyclictest0-21swapper/222:20:322
2328299342,14cyclictest7950irq/56-dwc_otg_21:05:472
23282993415,4cyclictest13614-21if_eth000:00:412
2328299341,23cyclictest30839-21timerandwakeup00:20:472
23284993317,12cyclictest8163-21ls19:55:493
23284993317,12cyclictest8163-21ls19:55:493
2328299337,4cyclictest7850irq/56-dwc_otg21:50:402
2328299337,16cyclictest0-21swapper/222:10:332
2328299334,5cyclictest7850irq/56-dwc_otg19:40:322
2328299329,12cyclictest0-21swapper/221:55:472
2328299326,14cyclictest0-21swapper/220:30:312
2328299325,6cyclictest0-21swapper/223:25:342
2328299325,13cyclictest8050irq/56-dwc_otg_19:15:512
2328299324,5cyclictest7850irq/56-dwc_otg00:30:322
226922323,22sleep10-21swapper/119:06:181
2328299319,4cyclictest16031-21latency_hist20:20:202
2328299317,14cyclictest0-21swapper/200:25:312
2328299316,11cyclictest8050irq/56-dwc_otg_19:20:382
2328299314,9cyclictest0-21swapper/200:16:292
2328299312,4cyclictest7850irq/56-dwc_otg00:35:482
23282993110,8cyclictest0-21swapper/220:40:332
2328299306,5cyclictest7950irq/56-dwc_otg_20:55:342
2328299304,5cyclictest7950irq/56-dwc_otg_20:35:442
254722290,4sleep231-21ksoftirqd/220:45:352
2328299294,4cyclictest0-21swapper/221:20:452
2328299291,19cyclictest17254-21timerwakeupswit20:20:472
2328299286,6cyclictest24865-21sh19:15:192
2328299285,5cyclictest7850irq/56-dwc_otg20:50:432
2328299282,11cyclictest0-21swapper/222:26:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional