You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 12:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Apr 23, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2707121330,5sleep0288499cyclictest05:36:060
2884999610,24cyclictest0-21swapper/005:05:010
268002950,2sleep3381ktimersoftd/306:45:483
2884998917,8cyclictest0-21swapper/008:20:580
2884998823,49cyclictest908-21runrttasks06:59:110
288499873,26cyclictest0-21swapper/007:16:100
288499831,70cyclictest30931-21ls06:50:480
95052820,4sleep39504-21ssh09:01:043
171432820,2sleep00-21swapper/007:11:300
241182800,6sleep2288699cyclictest06:41:202
2884997716,8cyclictest0-21swapper/009:05:590
2884997630,14cyclictest0-21swapper/009:26:030
2884997621,32cyclictest24242-21sh08:00:470
26452760,5sleep32639-21munin-run04:30:483
288499756,14cyclictest0-21swapper/009:15:530
288499741,4cyclictest29563-21ssh08:46:030
288499736,28cyclictest0-21swapper/006:36:000
288499731,21cyclictest0-21swapper/008:01:190
288499722,28cyclictest0-21swapper/009:11:100
288499721,4cyclictest6397-21ssh09:36:080
288499721,4cyclictest6397-21ssh09:36:080
309522710,1sleep20-21swapper/208:08:012
2884997137,16cyclictest0-21swapper/008:16:000
288499712,19cyclictest0-21swapper/008:40:330
288499711,18cyclictest0-21swapper/009:55:460
288499702,52cyclictest0-21swapper/004:36:010
288499692,20cyclictest31245-21/usr/sbin/munin06:51:080
288499691,28cyclictest0-21swapper/009:55:590
288499691,25cyclictest14954-21ssh06:31:100
288499691,19cyclictest12541-21ssh07:46:060
2884996816,7cyclictest0-21swapper/008:15:260
2884996722,4cyclictest7850irq/56-dwc_otg_08:51:430
288499672,23cyclictest0-21swapper/007:55:440
288499672,22cyclictest0-21swapper/006:14:360
288499672,22cyclictest0-21swapper/006:14:360
288499672,16cyclictest0-21swapper/007:04:410
2886996625,9cyclictest7850irq/56-dwc_otg_06:01:042
2884996621,5cyclictest0-21swapper/005:19:110
2884996612,13cyclictest0-21swapper/006:03:380
288499652,21cyclictest0-21swapper/007:33:300
2884996520,9cyclictest0-21swapper/009:47:010
2884996519,11cyclictest0-21swapper/007:10:310
2884996514,12cyclictest29479-21rm08:06:110
288499651,21cyclictest0-21swapper/005:46:020
277282650,5sleep1288599cyclictest09:22:201
2884996415,14cyclictest0-21swapper/007:27:520
2884996415,14cyclictest0-21swapper/007:27:520
288499641,16cyclictest0-21swapper/007:21:100
197162640,2sleep00-21swapper/008:34:080
288499632,21cyclictest0-21swapper/006:25:350
288499632,15cyclictest0-21swapper/007:36:130
288499631,30cyclictest0-21swapper/006:17:210
288499631,20cyclictest25824-21ssh09:20:590
288699622,19cyclictest7950irq/56-dwc_otg_07:56:002
288499622,19cyclictest0-21swapper/004:56:010
288499622,19cyclictest0-21swapper/004:56:010
28732620,1sleep00-21swapper/004:30:490
28732620,1sleep00-21swapper/004:30:490
2884996130,22cyclictest7850irq/56-dwc_otg_05:25:470
288499612,17cyclictest0-21swapper/009:42:000
2884996115,10cyclictest0-21swapper/009:30:590
2884996114,11cyclictest0-21swapper/006:42:460
288499595,9cyclictest0-21swapper/008:40:590
288499591,25cyclictest0-21swapper/005:40:580
288499591,14cyclictest0-21swapper/008:26:170
288499579,5cyclictest0-21swapper/004:50:590
247425711,6sleep00-21swapper/004:28:590
38452560,2sleep30-21swapper/308:15:153
288699566,13cyclictest8596-21ssh08:20:582
288799554,17cyclictest19815-21diskmemload07:15:593
288699558,19cyclictest0-21swapper/206:11:002
288699558,19cyclictest0-21swapper/206:11:002
288699554,6cyclictest9278-21ssh09:01:012
2884995525,2cyclictest0-21swapper/007:40:540
288499542,30cyclictest18649-21sed05:11:160
95692530,2sleep19568-21unixbench_singl04:46:151
288699531,10cyclictest28055-21ssh07:26:002
288699531,10cyclictest28055-21ssh07:26:002
288699521,12cyclictest7950irq/56-dwc_otg_04:45:592
288599523,7cyclictest0-21swapper/105:01:001
288499522,37cyclictest26533-21latency_hist05:35:480
288499511,28cyclictest9910-21missed_timers09:01:100
230792510,5sleep3288799cyclictest07:58:403
288699507,7cyclictest7950irq/56-dwc_otg_08:35:592
288699503,7cyclictest0-21swapper/208:41:002
288499507,22cyclictest5731-21wc08:56:080
288499502,27cyclictest8757-21fschecks_time04:46:050
288699491,8cyclictest7750irq/56-dwc_otg09:41:002
288499496,29cyclictest0-21swapper/005:56:010
288699487,25cyclictest0-21swapper/204:36:002
2885994827,13cyclictest7850irq/56-dwc_otg_09:11:011
2884994821,18cyclictest0-21swapper/005:10:120
2884994816,16cyclictest12617-21fschecks_count06:26:050
288699474,6cyclictest7950irq/56-dwc_otg_05:26:012
288699467,25cyclictest0-21swapper/205:41:012
288699465,9cyclictest0-21swapper/206:31:012
26732460,5sleep2288699cyclictest09:31:152
288799457,26cyclictest0-21swapper/306:26:013
288699457,24cyclictest0-21swapper/205:46:002
288699452,6cyclictest0-21swapper/205:31:012
2886994512,9cyclictest7950irq/56-dwc_otg_09:47:012
288599457,23cyclictest0-21swapper/105:36:011
288699446,25cyclictest0-21swapper/209:51:012
288699446,24cyclictest0-21swapper/207:40:532
18872440,2sleep30-21swapper/309:31:043
288799432,7cyclictest0-21swapper/306:16:003
288699437,26cyclictest0-21swapper/209:36:002
288699437,26cyclictest0-21swapper/209:36:002
288699436,4cyclictest7850irq/56-dwc_otg_09:26:012
288699433,8cyclictest31273-21ssh06:50:592
2886994313,16cyclictest7950irq/56-dwc_otg_08:46:052
288499431,17cyclictest674850wget04:41:010
297332420,1sleep10-21swapper/107:26:571
297332420,1sleep10-21swapper/107:26:571
288799427,25cyclictest0-21swapper/308:40:523
288799427,24cyclictest0-21swapper/307:06:003
2886994226,4cyclictest7750irq/56-dwc_otg05:06:142
2886994217,9cyclictest7750irq/56-dwc_otg08:04:232
288599427,22cyclictest0-21swapper/108:11:011
2885994213,4cyclictest23-21ksoftirqd/106:56:091
288499422,26cyclictest23484-21wc05:26:080
288699418,15cyclictest4423-21ssh08:15:522
288699417,27cyclictest0-21swapper/209:11:012
288699414,10cyclictest0-21swapper/204:40:582
2886994119,9cyclictest7950irq/56-dwc_otg_07:16:052
2886994116,10cyclictest7850irq/56-dwc_otg_07:10:242
288599418,6cyclictest21-21rcuc/106:00:591
2885994121,5cyclictest23-21ksoftirqd/108:54:121
288499411,23cyclictest0-21swapper/005:51:010
288499411,22cyclictest0-21swapper/006:10:130
288599405,13cyclictest0-21swapper/106:11:001
288599405,13cyclictest0-21swapper/106:11:001
288599404,32cyclictest7950irq/56-dwc_otg_08:16:001
258342400,1sleep10-21swapper/110:00:331
208702400,5sleep1288599cyclictest07:16:141
288799397,24cyclictest0-21swapper/305:36:013
288799395,4cyclictest0-21swapper/308:16:013
288699398,14cyclictest18575-21sh08:32:022
288699397,22cyclictest0-21swapper/205:05:012
288699396,11cyclictest7750irq/56-dwc_otg08:56:032
288599396,22cyclictest0-21swapper/107:51:011
23222393,28sleep20-21swapper/204:27:072
139702393,4sleep113976-21ssh07:08:261
65302380,4sleep1288599cyclictest04:40:481
288799389,19cyclictest0-21swapper/309:25:243
288799387,23cyclictest0-21swapper/307:21:013
288799387,22cyclictest0-21swapper/307:51:013
288799385,21cyclictest18528-21ssh06:36:023
288799384,18cyclictest7950irq/56-dwc_otg_08:21:133
288799381,28cyclictest21149-21ls08:36:023
2886993810,6cyclictest7750irq/56-dwc_otg07:10:542
2886993810,14cyclictest301ktimersoftd/206:56:152
2885993817,4cyclictest7750irq/56-dwc_otg08:36:151
14552387,15sleep10-21swapper/104:26:051
67702370,5sleep1288599cyclictest06:10:481
288799377,24cyclictest8100-21rm07:41:003
2887993716,4cyclictest908-21runrttasks09:18:563
288699378,14cyclictest7850irq/56-dwc_otg_07:31:142
288699377,4cyclictest0-21swapper/204:55:582
288699377,4cyclictest0-21swapper/204:55:582
288699377,23cyclictest0-21swapper/205:11:012
288599372,30cyclictest0-21swapper/108:20:581
288599371,7cyclictest7750irq/56-dwc_otg06:36:001
288599371,27cyclictest8920-21latency_hist09:00:481
288599371,25cyclictest13740-21ls05:00:481
288599371,25cyclictest13740-21ls05:00:481
288599371,17cyclictest20808-21fschecks_time07:56:061
288799368,20cyclictest908-21runrttasks07:15:333
288799364,17cyclictest24591-21ssh08:01:003
288799361,28cyclictest0-21swapper/309:11:543
288799361,27cyclictest0-21swapper/309:52:063
2887993611,15cyclictest13914-21diskmemload07:28:353
2887993611,15cyclictest13914-21diskmemload07:28:353
288699367,11cyclictest301ktimersoftd/206:06:152
288699365,15cyclictest27589-21/usr/sbin/munin05:36:142
288699365,13cyclictest7850irq/56-dwc_otg_06:47:552
288699362,27cyclictest7950irq/56-dwc_otg_09:55:592
288699362,24cyclictest0-21swapper/207:01:022
2886993617,5cyclictest29623-21sh09:25:472
2886993612,4cyclictest7950irq/56-dwc_otg_09:17:012
288699361,15cyclictest7950irq/56-dwc_otg_07:36:152
288599369,5cyclictest0-21swapper/109:48:151
288599368,5cyclictest7750irq/56-dwc_otg07:21:021
2885993615,4cyclictest21-21rcuc/105:35:471
2885993613,4cyclictest28654-21cpu05:41:021
288599361,25cyclictest0-21swapper/109:01:011
288799357,15cyclictest0-21swapper/304:51:013
288799356,4cyclictest0-21swapper/305:01:033
2887993522,9cyclictest3023-21cron06:00:483
288799352,12cyclictest0-21swapper/306:50:593
288799351,24cyclictest32720-21ls07:31:083
288799351,21cyclictest31855-21latency_hist05:50:483
288799351,18cyclictest17153-21latency_hist09:10:483
288699357,12cyclictest11358-21tail04:51:162
288699357,10cyclictest31-21ksoftirqd/206:20:162
288699355,15cyclictest7950irq/56-dwc_otg_06:00:482
288699354,9cyclictest7950irq/56-dwc_otg_08:54:462
288599352,7cyclictest29186-21ssh08:06:061
2885993519,5cyclictest23-21ksoftirqd/107:36:031
2885993518,5cyclictest15065-21sh06:31:121
2885993514,6cyclictest20199-21rm08:35:011
288599351,20cyclictest11164-21ls04:51:141
2885993510,5cyclictest0-21swapper/109:45:121
288799347,3cyclictest0-21swapper/308:49:333
288799341,22cyclictest29342-21ls08:06:093
288699346,8cyclictest7850irq/56-dwc_otg_08:11:092
288699345,17cyclictest7850irq/56-dwc_otg_05:22:262
288699344,20cyclictest1180-21seq05:55:302
288699344,16cyclictest7950irq/56-dwc_otg_07:53:102
288699344,11cyclictest27175-21ssh07:24:502
288699343,17cyclictest7950irq/56-dwc_otg_08:29:182
288699342,5cyclictest0-21swapper/204:30:572
288699342,5cyclictest0-21swapper/204:30:572
2886993418,10cyclictest7850irq/56-dwc_otg_09:08:142
288599348,17cyclictest1753-21wget09:31:031
288599346,6cyclictest4460-21cron04:35:481
288599346,6cyclictest4460-21cron04:35:481
288599345,20cyclictest8019-21ssh07:40:531
288599343,8cyclictest7469-21wget07:01:021
288599342,21cyclictest25188-21/usr/sbin/munin08:41:021
288599341,6cyclictest0-21swapper/105:56:001
2885993415,9cyclictest23-21ksoftirqd/108:01:061
2885993410,11cyclictest7750irq/56-dwc_otg08:46:161
288799338,13cyclictest0-21swapper/309:42:453
288799337,17cyclictest0-21swapper/308:56:013
288799333,4cyclictest0-21swapper/306:58:453
2887993325,5cyclictest9573-21latency_hist09:40:483
2887993325,5cyclictest9573-21latency_hist09:40:483
288799332,4cyclictest0-21swapper/307:40:343
288699338,5cyclictest7750irq/56-dwc_otg07:46:062
288599338,4cyclictest0-21swapper/109:51:031
288599335,6cyclictest16662-21cat08:30:481
288599331,5cyclictest31787-21irqstats06:51:071
2885993312,5cyclictest23-21ksoftirqd/105:06:111
288799327,23cyclictest0-21swapper/304:56:013
288799327,23cyclictest0-21swapper/304:56:013
288799327,15cyclictest0-21swapper/305:51:013
288799324,22cyclictest7950irq/56-dwc_otg_06:10:573
288799324,22cyclictest7950irq/56-dwc_otg_06:10:573
288799322,15cyclictest7950irq/56-dwc_otg_07:01:053
2887993212,6cyclictest0-21swapper/309:26:283
288799321,10cyclictest7950irq/56-dwc_otg_09:46:083
288699321,23cyclictest13914-21diskmemload06:30:472
288599327,14cyclictest7950irq/56-dwc_otg_07:47:231
288599326,4cyclictest7750irq/56-dwc_otg06:25:481
288599325,5cyclictest7750irq/56-dwc_otg07:31:131
2885993223,7cyclictest221ktimersoftd/106:40:581
2885993220,10cyclictest7950irq/56-dwc_otg_09:30:321
288599321,3cyclictest7750irq/56-dwc_otg07:11:111
288599321,14cyclictest0-21swapper/104:40:581
2885993210,6cyclictest7750irq/56-dwc_otg05:11:021
288799318,4cyclictest0-21swapper/309:57:533
288799314,14cyclictest7950irq/56-dwc_otg_05:06:083
288799314,14cyclictest7750irq/56-dwc_otg08:31:063
2887993125,4cyclictest17815-21fschecks_count05:11:053
2887993120,8cyclictest908-21runrttasks08:26:043
288699311,7cyclictest20926-21munin-run05:20:472
2886993113,5cyclictest13914-21diskmemload06:36:342
288599315,8cyclictest7750irq/56-dwc_otg09:20:471
288599314,17cyclictest0-21swapper/106:47:061
288599313,13cyclictest0-21swapper/105:25:591
2885993114,4cyclictest21251-21aten_rbpower_po05:21:041
2885993112,4cyclictest16398-21ssh09:09:251
2885993111,13cyclictest21-21rcuc/105:52:451
288799309,10cyclictest7850irq/56-dwc_otg_06:31:013
288799302,20cyclictest0-21swapper/305:32:253
288799302,20cyclictest0-21swapper/304:38:523
288599309,5cyclictest23-21ksoftirqd/109:36:151
288599309,5cyclictest23-21ksoftirqd/109:36:151
288599305,4cyclictest7750irq/56-dwc_otg06:16:021
288799299,4cyclictest0-21swapper/306:46:063
288799293,4cyclictest7950irq/56-dwc_otg_05:41:143
288799292,18cyclictest0-21swapper/305:30:163
288799291,8cyclictest25577-21systemd-journal08:55:483
2887992915,5cyclictest12584-21fschecks_time07:46:063
288699297,4cyclictest7950irq/56-dwc_otg_06:25:482
288599298,7cyclictest21-21rcuc/106:28:241
288599292,18cyclictest0-21swapper/105:16:451
65562281,4sleep30-21swapper/306:09:223
288799289,5cyclictest21567-21if_eth005:21:083
288799284,3cyclictest7950irq/56-dwc_otg_04:46:063
288799284,11cyclictest7850irq/56-dwc_otg_06:05:493
288799283,5cyclictest4490-21awk04:35:483
288799283,5cyclictest4490-21awk04:35:483
288799281,9cyclictest917-21snmpd06:23:363
288799263,11cyclictest7850irq/56-dwc_otg_05:19:203
2885992518,5cyclictest23-21ksoftirqd/105:46:001
288799241,4cyclictest7950irq/56-dwc_otg_04:41:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional