You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 22:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon May 04, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1914991092,81cyclictest19373-21/usr/sbin/munin20:36:050
125452960,3sleep312546-21modprobe17:00:533
191499933,64cyclictest0-21swapper/020:11:050
191499871,71cyclictest6823-21grep18:11:190
191499821,69cyclictest4474-21fschecks_time21:36:120
191499821,66cyclictest22927-21wc17:26:220
191499812,67cyclictest9961-21fschecks_count19:06:110
1914998121,45cyclictest21355-21/usr/sbin/munin18:41:040
1914997922,5cyclictest20191-21wget17:21:080
191499789,57cyclictest0-21swapper/020:51:160
191499781,71cyclictest31617-21latency_hist20:50:540
274302770,5sleep2191699cyclictest18:47:272
214652770,5sleep2191699cyclictest20:37:472
191499773,58cyclictest0-21swapper/018:21:050
1914997622,3cyclictest0-21swapper/017:36:080
191499756,15cyclictest0-21swapper/021:19:160
191499755,43cyclictest0-21swapper/019:51:060
191499752,63cyclictest0-21swapper/018:36:000
1914997521,42cyclictest7950irq/56-dwc_otg_19:39:540
1914997517,47cyclictest0-21swapper/019:14:340
191499739,45cyclictest0-21swapper/020:21:200
191499722,58cyclictest0-21swapper/019:46:150
275512710,2sleep30-21swapper/318:47:403
191499715,47cyclictest0-21swapper/021:26:050
1914997111,3cyclictest0-21swapper/020:31:070
191499702,53cyclictest15747-21unixbench_singl17:06:220
191499701,54cyclictest0-21swapper/019:20:040
1914997015,41cyclictest0-21swapper/017:15:540
1914997013,10cyclictest0-21swapper/019:22:320
1914997013,10cyclictest0-21swapper/019:22:320
191499701,17cyclictest0-21swapper/021:11:100
270922690,2sleep00-21swapper/020:06:080
191499692,6cyclictest26408-21ssh19:26:080
191499692,20cyclictest0-21swapper/020:05:220
1914996919,7cyclictest0-21swapper/020:45:010
191499682,58cyclictest0-21swapper/021:58:360
191499681,18cyclictest0-21swapper/021:53:320
191599675,22cyclictest7750irq/56-dwc_otg19:01:041
191499672,20cyclictest0-21swapper/021:44:360
191499671,56cyclictest0-21swapper/020:26:320
191499671,20cyclictest0-21swapper/017:33:450
263312660,2sleep10-21swapper/119:26:071
1914996626,32cyclictest25482-21ssh21:21:220
191499662,48cyclictest0-21swapper/019:01:210
191499662,24cyclictest0-21swapper/021:50:130
191499662,24cyclictest0-21swapper/021:50:130
1914996621,5cyclictest4801-21unixbench_singl20:56:230
1914996620,4cyclictest7950irq/56-dwc_otg_19:45:530
1914996615,37cyclictest3104-21ssh20:16:100
191499661,51cyclictest12865-21aten_rbpower_po17:01:070
1917996525,37cyclictest12694-21diskmemload19:16:053
191499657,44cyclictest0-21swapper/019:56:160
1914996512,5cyclictest5006-21chrt19:00:380
191499642,45cyclictest27367-21aten_rbpower_cu17:41:080
191499641,16cyclictest4586-21ntp_kernel_err18:06:170
191799634,6cyclictest23065-21unixbench_singl17:26:233
191499631,54cyclictest0-21swapper/021:06:110
191499631,54cyclictest0-21swapper/021:06:110
1914996310,9cyclictest15483-21ssh18:33:190
191799625,10cyclictest7950irq/56-dwc_otg_20:11:073
191799623,4cyclictest7950irq/56-dwc_otg_20:31:073
191599625,23cyclictest7750irq/56-dwc_otg17:06:051
191499627,7cyclictest2228-21fschecks_time16:31:110
1914996226,7cyclictest0-21swapper/018:51:050
191499622,15cyclictest0-21swapper/019:31:250
191499621,7cyclictest18446-21fschecks_count17:16:100
191499621,49cyclictest32509-21munin-plugin-st17:55:540
191499621,49cyclictest32509-21munin-plugin-st17:55:540
267082610,1sleep20-21swapper/221:23:362
191699617,11cyclictest7750irq/56-dwc_otg21:11:062
191699612,10cyclictest7750irq/56-dwc_otg19:26:052
191499619,11cyclictest0-21swapper/016:36:040
191499619,11cyclictest0-21swapper/016:36:040
191499612,13cyclictest0-21swapper/018:46:180
191799607,6cyclictest7950irq/56-dwc_otg_19:06:133
1917996019,38cyclictest15548-21ssh21:10:513
1917996019,38cyclictest15548-21ssh21:10:513
191499602,20cyclictest0-21swapper/021:33:220
191499602,20cyclictest0-21swapper/021:33:220
191799593,6cyclictest14968-21fschecks_time19:51:103
1917995910,41cyclictest0-21swapper/318:51:283
191499593,43cyclictest0-21swapper/016:46:100
191499591,43cyclictest9438-21cut16:51:110
191799584,4cyclictest7950irq/56-dwc_otg_17:15:543
1917995811,4cyclictest0-21swapper/319:32:153
1914995812,7cyclictest0-21swapper/021:05:110
191499581,15cyclictest0-21swapper/017:58:210
26582570,2sleep30-21swapper/320:54:593
191799578,34cyclictest20052-21ssh21:16:053
191799572,7cyclictest6859-21grep18:11:193
191599575,7cyclictest7750irq/56-dwc_otg16:51:051
191799562,4cyclictest7950irq/56-dwc_otg_21:36:143
191499561,45cyclictest2453-21grep18:01:120
191799554,4cyclictest7750irq/56-dwc_otg21:56:103
191799551,6cyclictest9950-21ssh19:45:313
191799551,4cyclictest7950irq/56-dwc_otg_21:26:203
191499556,10cyclictest0-21swapper/018:27:360
191799544,3cyclictest7950irq/56-dwc_otg_18:41:593
191799543,7cyclictest16655-21ssh21:51:053
191799541,5cyclictest24417-21diskstats21:21:093
191799541,5cyclictest0-21swapper/319:46:153
191799541,4cyclictest7950irq/56-dwc_otg_19:26:123
191499541,43cyclictest6370-21grep16:41:180
191499541,14cyclictest0-21swapper/017:50:060
191799532,3cyclictest0-21swapper/318:40:543
191799531,6cyclictest3196-21rm18:57:183
1917995316,34cyclictest3808-21idleruntime-cro21:35:543
1917995316,34cyclictest3808-21idleruntime-cro21:35:543
191799531,5cyclictest8422-21ntpq21:01:163
191799531,5cyclictest4778-21ssh20:56:233
191599532,12cyclictest0-21swapper/117:51:041
191599532,12cyclictest0-21swapper/117:51:041
191499532,19cyclictest0-21swapper/017:00:040
1914995312,7cyclictest0-21swapper/018:19:280
155742530,2sleep30-21swapper/321:49:343
155742530,2sleep30-21swapper/321:49:343
191799523,4cyclictest7850irq/56-dwc_otg_17:21:083
191799521,5cyclictest7750irq/56-dwc_otg21:11:123
1917995211,5cyclictest10082-21processes18:21:183
191599523,6cyclictest0-21swapper/118:16:051
191799513,3cyclictest7950irq/56-dwc_otg_19:36:063
191799512,4cyclictest0-21swapper/320:16:083
191599516,38cyclictest0-21swapper/117:26:071
191799502,4cyclictest7850irq/56-dwc_otg_20:30:543
191799501,7cyclictest24073-21ls20:01:223
191799501,5cyclictest23520-21apt-get20:41:053
191799501,5cyclictest11371-21sh21:44:363
191799501,4cyclictest7850irq/56-dwc_otg_20:08:143
191799501,4cyclictest19390-21apt-get20:36:063
191799501,3cyclictest0-21swapper/319:01:143
1917995012,4cyclictest17036-21latency_hist18:35:543
191799497,28cyclictest0-21swapper/316:46:073
191799495,6cyclictest15068-21kernelversion17:06:143
191799493,7cyclictest7950irq/56-dwc_otg_17:39:323
191799492,4cyclictest0-21swapper/320:23:483
191799492,3cyclictest0-21swapper/319:22:583
191799492,3cyclictest0-21swapper/319:22:583
1917994913,3cyclictest29404-21fschecks_count17:46:103
1917994910,5cyclictest109372chrt16:55:543
191599490,20cyclictest5240-21latency_hist19:00:541
191799487,6cyclictest11179-21basename18:26:083
191799484,4cyclictest7850irq/56-dwc_otg_19:14:343
191799482,4cyclictest7850irq/56-dwc_otg_19:59:403
191799481,4cyclictest0-21swapper/316:31:283
191799481,3cyclictest0-21swapper/317:16:033
191599488,13cyclictest7850irq/56-dwc_otg_17:21:051
191599487,28cyclictest0-21swapper/118:36:081
191599487,27cyclictest0-21swapper/121:06:071
191599487,27cyclictest0-21swapper/121:06:071
191699475,7cyclictest7950irq/56-dwc_otg_18:11:032
191699471,9cyclictest0-21swapper/217:06:062
191599478,25cyclictest0-21swapper/118:21:061
191599477,26cyclictest0-21swapper/117:01:071
191599475,15cyclictest24786-21sed20:41:221
191799469,6cyclictest31025-21aten_rbpower_po17:51:073
191799469,6cyclictest31025-21aten_rbpower_po17:51:073
191799461,4cyclictest0-21swapper/317:33:453
191699463,6cyclictest0-21swapper/217:16:042
191599466,29cyclictest30444-21ssh19:31:071
191599466,16cyclictest26473-21ssh20:05:311
1915994622,12cyclictest7950irq/56-dwc_otg_19:54:141
239432450,5sleep2191699cyclictest19:22:142
239432450,5sleep2191699cyclictest19:22:142
191799452,6cyclictest7950irq/56-dwc_otg_18:16:453
191699457,25cyclictest0-21swapper/217:11:062
191699456,24cyclictest0-21swapper/219:01:072
191599455,17cyclictest20820-21ntpq21:16:161
191599454,24cyclictest21547-21ssh19:20:251
191599452,26cyclictest8787-21/usr/sbin/munin21:41:131
1915994522,11cyclictest7950irq/56-dwc_otg_21:39:201
191799441,4cyclictest14072-21runrttasks17:02:413
191699447,26cyclictest0-21swapper/216:51:072
191699447,24cyclictest0-21swapper/218:06:092
191699444,9cyclictest7950irq/56-dwc_otg_20:01:062
191699442,5cyclictest7950irq/56-dwc_otg_21:56:062
191699441,15cyclictest12530-21ssh21:46:062
191699441,15cyclictest12530-21ssh21:46:062
191599448,24cyclictest0-21swapper/121:01:061
1915994413,15cyclictest7950irq/56-dwc_otg_18:41:201
1915994413,14cyclictest7950irq/56-dwc_otg_19:43:001
1915994412,18cyclictest7950irq/56-dwc_otg_21:11:071
191799437,22cyclictest0-21swapper/317:56:063
191599439,21cyclictest32200-21chrt20:11:201
191599438,18cyclictest0-21swapper/117:56:061
191599437,24cyclictest0-21swapper/117:36:071
191599437,20cyclictest32244-21sh21:30:541
191599432,10cyclictest0-21swapper/117:16:061
1915994316,13cyclictest7950irq/56-dwc_otg_20:48:001
191799421,6cyclictest30242-21ssh20:48:393
191699427,22cyclictest0-21swapper/218:51:062
191599429,12cyclictest17073-21ssh19:14:471
191599425,24cyclictest9704-21aten_rbpower_en19:06:081
191599425,12cyclictest6520-21awk18:11:161
191599422,6cyclictest7850irq/56-dwc_otg_16:46:061
1915994215,15cyclictest7750irq/56-dwc_otg20:16:381
1915994214,11cyclictest7750irq/56-dwc_otg19:25:421
1915994214,11cyclictest7750irq/56-dwc_otg19:25:421
1915994214,10cyclictest221ktimersoftd/121:56:041
1915994213,13cyclictest7950irq/56-dwc_otg_19:39:501
1915994212,15cyclictest7850irq/56-dwc_otg_20:56:551
191799419,5cyclictest25577-21systemd-journal16:38:353
191799419,5cyclictest25577-21systemd-journal16:38:353
191699417,25cyclictest0-21swapper/220:11:072
191699415,25cyclictest20070-21ssh21:16:072
1916994115,5cyclictest7750irq/56-dwc_otg18:30:542
191599415,25cyclictest12824-21rm19:48:131
1915994114,13cyclictest221ktimersoftd/121:46:201
1915994114,13cyclictest221ktimersoftd/121:46:201
1915994111,10cyclictest0-21swapper/117:34:061
158624121,13sleep00-21swapper/016:30:160
191799407,23cyclictest0-21swapper/317:41:083
191799401,13cyclictest7950irq/56-dwc_otg_18:03:383
191699407,23cyclictest0-21swapper/218:41:072
191699405,23cyclictest6290-21ssh19:40:592
1916994022,15cyclictest0-21swapper/219:31:082
191599409,11cyclictest15425-21ssh18:33:131
191599407,6cyclictest7850irq/56-dwc_otg_18:01:071
191599407,19cyclictest7950irq/56-dwc_otg_20:31:201
191599406,12cyclictest31370-21ssh18:52:071
191599405,25cyclictest7850irq/56-dwc_otg_18:46:071
1915994013,15cyclictest7950irq/56-dwc_otg_20:51:131
191699398,6cyclictest7750irq/56-dwc_otg21:51:052
191599397,15cyclictest20328-21ssh21:55:431
191599395,24cyclictest0-21swapper/119:56:071
191599392,21cyclictest26065-21ssh21:22:251
1915993917,5cyclictest30798-21sh20:10:521
1915993913,7cyclictest0-21swapper/117:11:151
191599391,28cyclictest27685-21ls17:41:121
191699387,22cyclictest0-21swapper/218:36:082
1916993822,5cyclictest31195-21fschecks_count17:51:102
1916993822,5cyclictest31195-21fschecks_count17:51:102
191699381,24cyclictest29987-21ls17:46:182
191599382,24cyclictest0-21swapper/116:56:071
1915993814,8cyclictest7750irq/56-dwc_otg18:06:141
191599381,24cyclictest0-21swapper/116:36:071
191599381,24cyclictest0-21swapper/116:36:071
191799373,4cyclictest7850irq/56-dwc_otg_16:41:493
1916993715,4cyclictest4887-21grep16:36:222
1916993715,4cyclictest4887-21grep16:36:222
1916993710,7cyclictest7850irq/56-dwc_otg_19:10:542
191599376,13cyclictest0-21swapper/120:29:071
191599373,4cyclictest7950irq/56-dwc_otg_20:21:051
1915993712,12cyclictest7850irq/56-dwc_otg_16:34:181
1915993710,20cyclictest221ktimersoftd/120:36:351
9992360,2sleep20-21swapper/216:26:202
191699361,20cyclictest0-21swapper/216:41:072
191599369,20cyclictest32349-21ssh21:30:551
191599369,20cyclictest32349-21ssh21:30:551
191599364,18cyclictest12414-21taskset18:27:301
191599362,16cyclictest7850irq/56-dwc_otg_17:46:191
10802369,11sleep30-21swapper/316:26:223
191699357,20cyclictest7950irq/56-dwc_otg_21:41:042
191699351,27cyclictest14200-21ssh18:31:222
191699351,23cyclictest27319-21kworker/u8:220:58:022
1916993512,3cyclictest0-21swapper/219:36:052
1916993510,11cyclictest7950irq/56-dwc_otg_18:56:232
191599356,5cyclictest7950irq/56-dwc_otg_16:45:541
191799341,3cyclictest0-21swapper/318:09:373
191699346,15cyclictest7950irq/56-dwc_otg_21:26:212
191699345,10cyclictest7850irq/56-dwc_otg_20:31:182
191699337,13cyclictest0-21swapper/221:36:222
191699334,5cyclictest7850irq/56-dwc_otg_20:06:052
191699331,22cyclictest0-21swapper/218:04:262
191699328,5cyclictest31-21ksoftirqd/217:26:232
191699327,17cyclictest7950irq/56-dwc_otg_21:06:222
191699327,17cyclictest7950irq/56-dwc_otg_21:06:222
191699327,15cyclictest0-21swapper/217:21:072
191699326,6cyclictest7850irq/56-dwc_otg_17:01:112
191699326,16cyclictest0-21swapper/219:59:212
191699325,14cyclictest7950irq/56-dwc_otg_18:21:212
191699322,5cyclictest7850irq/56-dwc_otg_20:46:062
1916993225,4cyclictest6672-21sh20:20:532
1916993223,6cyclictest18160-21sh19:16:082
1916993210,5cyclictest27101-21idleruntime-cro17:40:542
191699317,12cyclictest0-21swapper/219:15:362
191699311,4cyclictest7950irq/56-dwc_otg_16:46:212
191699311,14cyclictest7850irq/56-dwc_otg_17:35:542
191699307,4cyclictest7950irq/56-dwc_otg_19:46:222
191699307,4cyclictest0-21swapper/220:51:132
191699306,4cyclictest7850irq/56-dwc_otg_20:26:112
191699305,7cyclictest611-21sort17:56:102
191699303,17cyclictest31-21ksoftirqd/219:51:122
191699303,16cyclictest28474-21unixbench_singl17:41:222
1916993010,4cyclictest0-21swapper/221:31:502
1916993010,4cyclictest0-21swapper/221:31:502
1916993010,4cyclictest0-21swapper/221:01:422
191699292,5cyclictest0-21swapper/220:40:582
191699287,3cyclictest0-21swapper/216:56:102
1916992815,4cyclictest8107-21sh20:21:212
13622283,15sleep10-21swapper/116:27:261
191699274,4cyclictest7511-21/usr/sbin/munin18:16:162
191699264,4cyclictest7850irq/56-dwc_otg_16:31:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional