You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 06:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Jul 28, 2026 00:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
861721190,2sleep30-21swapper/300:02:553
18121991142,76cyclictest14096-21ssh22:10:471
18121991134,46cyclictest7950irq/56-dwc_otg_00:20:261
181209911311,81cyclictest0-21swapper/022:35:370
181209911132,60cyclictest22443-21ssh00:20:270
181209910910,78cyclictest0-21swapper/021:10:440
2323421080,6sleep31812399cyclictest00:20:473
18120991086,64cyclictest0-21swapper/000:30:300
181209910738,20cyclictest5119-21sh21:20:380
18121991056,72cyclictest22369-21fschecks_count23:00:431
18121991035,14cyclictest0-21swapper/123:25:551
18122991023,10cyclictest0-21swapper/200:15:532
18121991002,16cyclictest8050irq/56-dwc_otg_23:20:531
181219910013,68cyclictest7850irq/56-dwc_otg00:25:421
181219910013,68cyclictest7850irq/56-dwc_otg00:25:421
1812199993,75cyclictest15195-21ssh00:10:511
1812099978,29cyclictest22428-21ssh23:00:440
1812099972,91cyclictest0-21swapper/022:40:530
18121999613,8cyclictest8050irq/56-dwc_otg_00:30:481
1812099968,60cyclictest0-21swapper/023:20:530
18120999370,9cyclictest4690-21kworker/0:222:10:450
1812199929,67cyclictest8050irq/56-dwc_otg_00:00:501
18122998911,18cyclictest8050irq/56-dwc_otg_00:30:422
323632880,5sleep31812399cyclictest23:51:563
18120998810,64cyclictest22918-21fschecks_count00:20:430
59142870,1sleep20-21swapper/200:39:512
1812099872,56cyclictest15253-21proc_pri00:10:520
12522860,5sleep31812399cyclictest21:55:383
18120998517,46cyclictest11232-21ls23:25:550
1812099834,45cyclictest27010-21diskmemload00:25:440
1812099834,45cyclictest27010-21diskmemload00:25:440
1812199812,11cyclictest29788-21ls21:10:441
18120998047,7cyclictest7850irq/56-dwc_otg21:55:400
1812099800,47cyclictest30254-21latency_hist23:50:250
1812299792,56cyclictest27268-21ls00:25:492
1812299792,56cyclictest27268-21ls00:25:492
1812199783,15cyclictest1552-21ssh22:35:381
1812099783,63cyclictest3389-21unixbench_multi23:55:550
1812099783,63cyclictest3389-21unixbench_multi23:55:550
1812099783,45cyclictest0-21swapper/000:00:500
150052770,5sleep21812299cyclictest21:31:412
150052770,5sleep21812299cyclictest21:31:412
1812099757,50cyclictest0-21swapper/021:02:200
18120997543,5cyclictest0-21swapper/022:50:380
1812099736,49cyclictest29005-21cron21:10:240
1812099735,53cyclictest14566-21cron20:30:250
1812099733,47cyclictest0-21swapper/022:25:390
1812299723,44cyclictest11267-21timerwakeupswit23:25:562
18120997210,42cyclictest26005-21if_wlan021:45:420
1812099714,44cyclictest28855-21latency_hist19:40:240
18120997133,8cyclictest0-21swapper/022:05:390
1812399702,43cyclictest14050-21ls22:10:453
1812299703,23cyclictest31-21ksoftirqd/223:00:432
18120997037,6cyclictest0-21swapper/020:15:360
18123996912,33cyclictest0-21swapper/323:20:543
1812099696,44cyclictest0-21swapper/019:25:380
18120996925,6cyclictest0-21swapper/022:15:390
18120996910,42cyclictest30948-21aten_rbpower_en19:45:380
1812199681,6cyclictest23825-21grep00:20:561
18120996720,31cyclictest4623-21wc20:00:510
1812099664,43cyclictest0-21swapper/020:41:230
36032650,2sleep20-21swapper/221:18:122
36032650,2sleep20-21swapper/221:18:122
1812199651,6cyclictest17654-21timerwakeupswit20:35:531
1812199651,6cyclictest17654-21timerwakeupswit20:35:531
18120996546,5cyclictest0-21swapper/023:55:250
1812099645,39cyclictest16664-21basename20:35:390
1812099645,39cyclictest16664-21basename20:35:390
176152641,14sleep20-21swapper/219:07:212
1812099637,15cyclictest15472-21unixbench_singl23:30:560
1812099637,15cyclictest15472-21unixbench_singl23:30:560
1812099636,39cyclictest22225-21ssh22:20:430
1812099630,39cyclictest1261-21sh21:15:420
1812099630,39cyclictest1261-21sh21:15:420
163442630,1sleep30-21swapper/321:34:073
163442630,1sleep30-21swapper/321:34:073
122452630,2sleep20-21swapper/200:07:012
1812399628,29cyclictest39-21ksoftirqd/321:10:453
1812199622,4cyclictest0-21swapper/122:50:541
1812199621,7cyclictest2008-21switchtime21:15:521
1812199621,7cyclictest2008-21switchtime21:15:521
1812199621,6cyclictest30214-21unixbench_multi19:40:541
18121996212,8cyclictest505-21cron21:25:241
1812099628,23cyclictest21862-21/usr/sbin/munin21:40:430
18120996231,23cyclictest9851-21missed_timers21:25:480
18120996231,19cyclictest0-21swapper/020:05:370
18120996231,19cyclictest0-21swapper/020:05:370
18120996220,27cyclictest18616-21unixbench_multi21:35:550
1812199613,6cyclictest26096-21cron23:45:241
1812199613,5cyclictest0-21swapper/121:45:311
1812199613,11cyclictest7850irq/56-dwc_otg23:20:251
1812099619,4cyclictest0-21swapper/022:55:400
1812099618,18cyclictest0-21swapper/022:49:430
1812099618,18cyclictest0-21swapper/022:49:430
1812099612,50cyclictest16958-21ssh21:35:140
1812099612,50cyclictest16958-21ssh21:35:140
1812099612,34cyclictest0-21swapper/019:16:080
1812099612,34cyclictest0-21swapper/019:16:080
18122996015,21cyclictest0-21swapper/222:10:472
1812199602,8cyclictest18823-21kernelversion23:35:471
1812199601,8cyclictest8050irq/56-dwc_otg_22:19:121
1812199601,6cyclictest413-21rm23:13:201
1812099606,45cyclictest0-21swapper/021:53:580
1812099606,45cyclictest0-21swapper/021:53:580
1812099602,29cyclictest588650rm23:20:260
1812399596,33cyclictest8050irq/56-dwc_otg_00:20:263
1812399594,27cyclictest8050irq/56-dwc_otg_00:30:493
1812199592,6cyclictest27469-21ls23:45:541
1812199592,6cyclictest10067-21taskset22:45:411
1812199592,6cyclictest10067-21taskset22:45:411
1812199591,6cyclictest30469-21kernelversion22:30:471
1812199591,11cyclictest7950irq/56-dwc_otg_21:40:511
1812099590,44cyclictest0-21swapper/022:32:520
18123995810,30cyclictest8050irq/56-dwc_otg_00:25:443
18123995810,30cyclictest8050irq/56-dwc_otg_00:25:443
1812199581,6cyclictest30520-21ssh23:50:371
18121995816,6cyclictest14111-21open_files21:30:501
18121995816,6cyclictest14111-21open_files21:30:501
1812099581,5cyclictest20595-21ls20:45:440
1812199572,4cyclictest0-21swapper/121:35:431
1812099579,35cyclictest14090-21ssh00:10:250
1812099577,46cyclictest27176-21latency_hist19:35:250
1812099576,34cyclictest0-21swapper/022:05:080
18120995713,27cyclictest24295-21sh23:42:050
1812399563,21cyclictest22352-21ssh23:00:433
1812199566,9cyclictest7850irq/56-dwc_otg21:29:521
1812199566,12cyclictest0-21swapper/123:35:191
1812199566,12cyclictest0-21swapper/123:35:191
1812199565,9cyclictest8050irq/56-dwc_otg_22:05:451
1812199561,6cyclictest125-21systemd-journal19:16:121
1812199561,6cyclictest125-21systemd-journal19:16:121
1812099568,32cyclictest3690-21tail00:35:540
1812099564,42cyclictest0-21swapper/023:12:020
1812099562,36cyclictest0-21swapper/019:55:370
1812099561,51cyclictest9-21ksoftirqd/023:35:520
259492550,5sleep21812299cyclictest00:24:412
1812399553,26cyclictest7950irq/56-dwc_otg_23:25:543
18122995516,18cyclictest31-21ksoftirqd/221:10:442
1812199553,4cyclictest1393-21basename21:55:411
1812199551,4cyclictest7850irq/56-dwc_otg22:00:421
18120995529,4cyclictest0-21swapper/020:50:370
1812199545,4cyclictest7850irq/56-dwc_otg19:30:251
1812199541,14cyclictest0-21swapper/123:57:431
1812199541,14cyclictest0-21swapper/123:57:431
1812099547,43cyclictest29754-21ntp_kernel_err19:40:480
1812099546,34cyclictest18491-21taskset19:10:440
1812099542,28cyclictest15963-21unixbench_singl20:30:540
35382530,3sleep20-21swapper/223:56:022
35382530,3sleep20-21swapper/223:56:022
1812299537,29cyclictest8050irq/56-dwc_otg_23:20:532
1812199532,15cyclictest7950irq/56-dwc_otg_20:30:541
1812099539,32cyclictest0-21swapper/019:53:370
18120995325,12cyclictest7850irq/56-dwc_otg19:20:410
1812299528,27cyclictest0-21swapper/219:16:092
1812299528,27cyclictest0-21swapper/219:16:092
1812199527,4cyclictest0-21swapper/123:07:351
1812199522,4cyclictest7850irq/56-dwc_otg22:55:491
1812199521,13cyclictest0-21swapper/121:52:361
1812199521,13cyclictest0-21swapper/121:52:361
1812099522,32cyclictest0-21swapper/020:10:390
1812199512,5cyclictest23167-21/usr/sbin/munin20:50:531
1812199512,10cyclictest0-21swapper/122:44:091
1812099514,35cyclictest0-21swapper/023:06:080
1812299503,20cyclictest0-21swapper/200:10:522
1812199502,7cyclictest0-21swapper/119:30:371
18120995033,10cyclictest11400-21ls20:20:400
1812399494,24cyclictest7950irq/56-dwc_otg_22:35:403
18123994915,17cyclictest8050irq/56-dwc_otg_00:10:493
1812299498,9cyclictest7850irq/56-dwc_otg22:35:382
1812299493,10cyclictest7033-21ntp_states00:00:502
1812199492,3cyclictest0-21swapper/122:20:551
1812199487,3cyclictest0-21swapper/100:10:041
280902471,4sleep228109-21ssh23:46:362
1812299478,25cyclictest0-21swapper/221:25:402
1812099479,5cyclictest24928-21unixbench_multi20:55:540
175152472,14sleep00-21swapper/019:06:050
1812299468,28cyclictest0-21swapper/221:40:372
1812299467,25cyclictest0-21swapper/220:45:402
1812199467,30cyclictest0-21swapper/119:35:391
1812199467,25cyclictest0-21swapper/121:00:391
1812199463,40cyclictest0-21swapper/120:50:241
84122450,5sleep21812299cyclictest21:24:322
312762451,38sleep331284-21rm21:51:323
312762451,38sleep331284-21rm21:51:323
1812299457,24cyclictest0-21swapper/219:25:292
18123994421,13cyclictest0-21swapper/320:55:393
1812299443,6cyclictest30796-21fschecks_count23:50:422
1812399437,22cyclictest0-21swapper/320:15:383
1812399433,9cyclictest17536-21ssh22:15:373
1812299421,27cyclictest9556-21ls22:45:252
1812199428,25cyclictest0-21swapper/120:10:391
1812199427,5cyclictest7850irq/56-dwc_otg00:35:401
1812199421,32cyclictest26228-21ls22:25:451
168922424,24sleep10-21swapper/119:05:501
1812399416,21cyclictest0-21swapper/322:20:313
1812299415,9cyclictest0-21swapper/222:55:402
1812299411,26cyclictest15006-21ssh22:50:542
1812199416,25cyclictest0-21swapper/120:40:381
18121994135,4cyclictest221ktimersoftd/120:15:361
180792412,33sleep20-21swapper/221:35:482
1812399404,19cyclictest8050irq/56-dwc_otg_23:05:363
18123994023,5cyclictest2739-21sh23:15:493
1812399401,26cyclictest0-21swapper/322:45:403
1812399401,26cyclictest0-21swapper/322:45:403
1812399393,4cyclictest8050irq/56-dwc_otg_19:30:373
18123993916,5cyclictest15078-21ssh22:50:543
1812299399,5cyclictest7950irq/56-dwc_otg_22:25:462
1812299388,18cyclictest0-21swapper/222:20:312
1812299383,4cyclictest0-21swapper/220:00:372
1812199387,17cyclictest0-21swapper/120:05:381
1812199387,17cyclictest0-21swapper/120:05:381
1812199384,24cyclictest0-21swapper/121:05:401
18123993715,8cyclictest750-21snmpd00:39:023
18123993710,15cyclictest0-21swapper/323:40:563
1812299379,15cyclictest8050irq/56-dwc_otg_22:05:542
1812399364,10cyclictest8050irq/56-dwc_otg_22:55:463
1812299362,4cyclictest0-21swapper/223:05:372
18122993612,5cyclictest0-21swapper/223:30:522
18122993612,5cyclictest0-21swapper/223:30:522
18121993617,6cyclictest24947-21grep20:55:541
1812399357,14cyclictest7850irq/56-dwc_otg23:10:393
1812399353,16cyclictest8050irq/56-dwc_otg_23:31:113
1812399353,16cyclictest8050irq/56-dwc_otg_23:31:113
1812399353,14cyclictest7950irq/56-dwc_otg_23:55:473
1812399353,14cyclictest7950irq/56-dwc_otg_23:55:473
1812399352,19cyclictest18716-21vmstat21:35:563
1812399352,16cyclictest8050irq/56-dwc_otg_22:30:513
18123993516,7cyclictest29025-21diskmemload23:45:473
18123993512,5cyclictest9480-21modprobe22:45:243
1812399351,24cyclictest5535-21ls20:05:253
1812399351,21cyclictest0-21swapper/322:30:253
1812299357,7cyclictest7850irq/56-dwc_otg23:35:542
18122993527,6cyclictest23955-21ssh23:41:262
1812299352,3cyclictest0-21swapper/220:31:092
1812399345,6cyclictest7850irq/56-dwc_otg21:40:423
1812399342,8cyclictest8050irq/56-dwc_otg_23:35:573
18123993412,5cyclictest0-21swapper/322:02:393
1812299348,4cyclictest0-21swapper/222:04:262
18122993422,8cyclictest16940-21fschecks_time20:35:432
18122993422,8cyclictest16940-21fschecks_time20:35:432
1812299341,22cyclictest7232-21latency_hist20:10:262
1812299341,22cyclictest7232-21latency_hist20:10:262
18122993410,4cyclictest0-21swapper/222:15:482
1812199343,6cyclictest0-21swapper/119:50:361
1812399337,17cyclictest39-21ksoftirqd/319:45:493
1812399332,6cyclictest27722-21fschecks_count21:05:433
1812399332,22cyclictest0-21swapper/319:35:493
18123993310,6cyclictest10588-21unixbench-2d22:05:543
1812299337,14cyclictest0-21swapper/220:50:382
1812299336,6cyclictest7950irq/56-dwc_otg_22:45:432
1812299336,6cyclictest7950irq/56-dwc_otg_22:45:432
1812299336,5cyclictest7950irq/56-dwc_otg_19:10:542
18122993315,5cyclictest29436-21latency_hist21:50:262
1812199334,16cyclictest8050irq/56-dwc_otg_19:25:241
1812399327,6cyclictest8050irq/56-dwc_otg_20:30:383
1812399326,13cyclictest7950irq/56-dwc_otg_21:02:433
1812399325,5cyclictest0-21swapper/321:19:453
1812399325,5cyclictest0-21swapper/321:19:453
1812399324,2cyclictest0-21swapper/319:50:363
1812399322,12cyclictest8050irq/56-dwc_otg_19:16:123
1812399322,12cyclictest8050irq/56-dwc_otg_19:16:123
18123993215,6cyclictest13537-1kworker/u9:221:45:453
1812399321,14cyclictest8050irq/56-dwc_otg_21:25:253
1812299328,5cyclictest7950irq/56-dwc_otg_20:10:372
1812299322,8cyclictest7950irq/56-dwc_otg_19:50:422
1812299321,8cyclictest18949-21latency20:40:442
18122993210,12cyclictest0-21swapper/221:50:342
18122993210,12cyclictest0-21swapper/221:50:342
1812199324,13cyclictest8050irq/56-dwc_otg_19:55:511
1812199322,14cyclictest8050irq/56-dwc_otg_19:45:521
18121993216,6cyclictest18582-21iwlist19:10:451
38802311,3sleep10-21swapper/120:00:391
1812399319,3cyclictest0-21swapper/300:06:123
1812399311,20cyclictest3078-21timerwakeupswit19:55:543
1812399311,17cyclictest8050irq/56-dwc_otg_21:25:443
1812399311,16cyclictest12770-21latency_hist20:25:253
1812299313,6cyclictest7950irq/56-dwc_otg_19:40:402
1812299313,19cyclictest2241-21ls21:55:532
1812299312,17cyclictest0-21swapper/220:25:402
1812299311,7cyclictest25719-21fschecks_count19:30:422
1812299311,20cyclictest0-21swapper/222:30:422
1812299311,15cyclictest7850irq/56-dwc_otg21:00:492
1812399306,13cyclictest8050irq/56-dwc_otg_20:45:243
1812399304,6cyclictest7950irq/56-dwc_otg_19:20:533
1812399302,18cyclictest6306-21grep20:05:463
1812399302,18cyclictest6306-21grep20:05:463
1812399301,14cyclictest8050irq/56-dwc_otg_20:35:533
1812399301,14cyclictest8050irq/56-dwc_otg_20:35:533
1812299309,10cyclictest7950irq/56-dwc_otg_20:15:512
1812299307,4cyclictest0-21swapper/223:18:382
1812299307,14cyclictest0-21swapper/219:45:382
1812299306,15cyclictest0-21swapper/219:55:402
1812299305,4cyclictest8050irq/56-dwc_otg_21:05:502
1812299301,4cyclictest0-21swapper/223:14:392
1812299301,26cyclictest0-21swapper/219:20:372
1812399296,16cyclictest0-21swapper/320:45:373
1812399292,5cyclictest8050irq/56-dwc_otg_19:10:473
1812299291,17cyclictest0-21swapper/220:55:402
1812299291,15cyclictest8050irq/56-dwc_otg_20:25:202
1812199291,5cyclictest11523-21/usr/sbin/munin20:20:431
1812399286,4cyclictest8050irq/56-dwc_otg_19:25:533
1812399284,13cyclictest8050irq/56-dwc_otg_20:50:463
1812399282,14cyclictest0-21swapper/320:10:423
1812299282,18cyclictest0-21swapper/219:39:362
1812199285,5cyclictest7850irq/56-dwc_otg20:26:261
1812399271,18cyclictest0-21swapper/320:25:453
1812399263,11cyclictest7950irq/56-dwc_otg_19:40:463
170112256,10sleep30-21swapper/319:05:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional