You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 08:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Nov 21, 2025 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1788923240,2sleep00-21swapper/019:05:340
186469914342,97cyclictest1120-21runrttasks23:10:190
65172950,5sleep013-21rcuc/022:50:340
18646999217,39cyclictest0-21swapper/019:40:160
1864699921,31cyclictest26763-21grep00:00:360
18646998420,15cyclictest29177-21unixbench_singl21:05:340
1864699803,60cyclictest0-21swapper/020:00:190
1864699801,27cyclictest1857-21aten_rbpower_en22:45:200
18646997727,4cyclictest7850irq/56-dwc_otg_00:25:350
18646997720,6cyclictest7850irq/56-dwc_otg_20:30:060
242792760,2sleep20-21swapper/220:55:072
1864699761,25cyclictest12674-21fschecks_time21:30:260
1864699751,16cyclictest935-21ssh23:27:000
224202740,2sleep339-21ksoftirqd/323:12:193
18646997422,9cyclictest7850irq/56-dwc_otg_23:35:360
1864699741,14cyclictest0-21swapper/000:35:260
1864699735,14cyclictest0-21swapper/019:30:270
18646997316,2cyclictest0-21swapper/022:25:370
88832720,3sleep15442-1kworker/u9:023:38:141
231832720,3sleep121-21rcuc/120:50:311
231832720,3sleep121-21rcuc/120:50:311
1864699721,23cyclictest0-21swapper/022:20:190
178002720,1sleep20-21swapper/223:05:382
18646997019,13cyclictest0-21swapper/023:21:390
18646997011,3cyclictest7950irq/56-dwc_otg_21:55:180
1864699682,20cyclictest15797-21chrt23:46:420
18646996821,11cyclictest0-21swapper/020:35:310
18646996815,6cyclictest17365-21sshd21:35:580
9882670,2sleep10-21swapper/122:44:071
9882670,2sleep10-21swapper/122:44:071
33542670,3sleep31864999cyclictest22:46:113
18646996740,19cyclictest0-21swapper/019:15:110
18646996740,19cyclictest0-21swapper/019:15:110
18646996715,11cyclictest0-21swapper/021:44:150
18646996715,11cyclictest0-21swapper/021:44:150
18646996713,13cyclictest0-21swapper/022:15:060
1864699671,28cyclictest0-21swapper/022:17:310
1864699671,22cyclictest0-21swapper/000:08:560
1864699671,17cyclictest0-21swapper/019:38:560
1864699671,15cyclictest0-21swapper/023:05:310
18646996711,13cyclictest0-21swapper/021:00:050
1864699661,22cyclictest0-21swapper/021:49:220
18646996612,19cyclictest0-21swapper/021:00:250
1864699651,4cyclictest17382-21diskmemload00:33:520
1864699651,25cyclictest0-21swapper/023:16:030
1864699651,22cyclictest0-21swapper/000:24:240
1864699646,13cyclictest0-21swapper/021:26:140
1864699636,41cyclictest0-21swapper/019:10:200
1864699631,5cyclictest26106-21ssh22:34:130
18646996314,4cyclictest7950irq/56-dwc_otg_23:30:230
18646996314,4cyclictest7950irq/56-dwc_otg_23:30:230
1864699631,21cyclictest22557-21ntp_states23:55:310
18646996240,7cyclictest7850irq/56-dwc_otg_20:55:060
18646996240,7cyclictest7850irq/56-dwc_otg_20:55:060
18646996212,11cyclictest0-21swapper/023:52:080
1864699617,20cyclictest0-21swapper/019:46:400
1864699612,32cyclictest31736-21ssh22:40:380
1864699612,32cyclictest31736-21ssh22:40:380
1864699611,17cyclictest0-21swapper/021:23:390
1864699601,20cyclictest0-21swapper/022:56:040
1864699591,21cyclictest0-21swapper/022:37:000
1864699591,15cyclictest12963-21basename23:00:190
1864699591,15cyclictest12963-21basename23:00:190
18646995731,5cyclictest7750irq/56-dwc_otg22:05:230
18646995731,5cyclictest7750irq/56-dwc_otg22:05:230
40912560,2sleep30-21swapper/321:18:313
1864699562,15cyclictest0-21swapper/000:14:490
1864699552,32cyclictest18953-21aten_rbpower_vo20:40:230
1864699551,29cyclictest4163-21idleruntime-cro20:00:060
1864699551,29cyclictest0-21swapper/020:49:230
1864699549,25cyclictest0-21swapper/021:51:170
1864699541,34cyclictest11358-21grep23:40:360
1864899537,26cyclictest0-21swapper/221:05:112
1864699532,27cyclictest0-21swapper/000:19:200
18646995322,17cyclictest2211-21ps21:15:310
1864699521,27cyclictest0-21swapper/019:50:270
1864699520,38cyclictest25591-21kworker/0:019:30:050
18646995011,25cyclictest0-21swapper/020:15:210
1864699492,25cyclictest15981-21ps20:30:360
1864799481,14cyclictest0-21swapper/122:55:211
1864699482,27cyclictest30774-21ps21:10:310
1864699482,27cyclictest30774-21ps21:10:310
1864699481,3cyclictest0-21swapper/019:20:250
1864999477,22cyclictest0-21swapper/319:55:203
1864799472,4cyclictest0-21swapper/121:35:181
68462460,2sleep30-21swapper/300:19:223
1864999467,25cyclictest0-21swapper/320:35:193
1864999467,25cyclictest0-21swapper/319:30:203
1864999467,25cyclictest0-21swapper/319:15:113
1864999467,25cyclictest0-21swapper/319:15:113
64702450,2sleep36463-21ssh22:50:333
1864999457,23cyclictest0-21swapper/320:45:223
1864899458,23cyclictest0-21swapper/219:30:202
1864899447,23cyclictest0-21swapper/220:35:192
1864899447,22cyclictest0-21swapper/219:55:202
18646994422,9cyclictest0-21swapper/020:10:310
1864999437,24cyclictest0-21swapper/322:05:193
1864999437,24cyclictest0-21swapper/322:05:193
1864999437,23cyclictest0-21swapper/319:25:193
1864999431,12cyclictest20207-21ls22:25:283
1864799436,25cyclictest0-21swapper/120:55:111
18647994315,6cyclictest20373-21ntp_kernel_pll_22:25:301
18646994315,15cyclictest1595-21ssh22:00:200
1864899427,23cyclictest0-21swapper/219:15:112
1864899427,23cyclictest0-21swapper/219:15:112
18646994215,6cyclictest0-21swapper/020:20:180
1864899417,23cyclictest0-21swapper/219:25:192
1864899417,20cyclictest0-21swapper/220:45:222
18648994118,6cyclictest9039-21sh00:21:122
1681824121,11sleep216824-21/usr/sbin/munin19:05:172
238662400,2sleep30-21swapper/323:57:163
1864899409,3cyclictest0-21swapper/223:44:122
18648994031,7cyclictest7950irq/56-dwc_otg_00:00:172
18648994017,6cyclictest7525-21sh23:35:412
18647994029,7cyclictest23-21ksoftirqd/123:55:191
18647994026,9cyclictest23-21ksoftirqd/122:10:241
1864699402,23cyclictest0-21swapper/020:05:360
1864699402,23cyclictest0-21swapper/020:05:360
1864999397,24cyclictest0-21swapper/319:45:183
1864899397,6cyclictest0-21swapper/223:00:192
1864899397,6cyclictest0-21swapper/223:00:192
18648993913,8cyclictest0-21swapper/220:40:292
1864799396,12cyclictest7750irq/56-dwc_otg23:45:181
1864799395,14cyclictest7850irq/56-dwc_otg_19:45:061
1864799391,6cyclictest29150-21ssh00:05:181
18647993912,7cyclictest7950irq/56-dwc_otg_23:40:191
235762381,27sleep20-21swapper/223:14:492
1864999385,22cyclictest0-21swapper/323:40:203
1864999381,8cyclictest0-21swapper/321:25:223
18648993812,5cyclictest1947-21/usr/sbin/munin19:50:332
18648993811,7cyclictest9060-21grep20:10:372
1864799387,22cyclictest0-21swapper/121:00:201
1864799383,5cyclictest0-21swapper/119:45:181
1864799382,6cyclictest0-21swapper/100:15:211
1864799381,8cyclictest0-21swapper/121:10:161
1864799381,8cyclictest0-21swapper/121:10:161
1864799381,5cyclictest7750irq/56-dwc_otg00:00:171
210602370,5sleep11864799cyclictest21:41:071
210602370,5sleep11864799cyclictest21:41:071
195892370,1sleep20-21swapper/222:25:162
1864999374,23cyclictest0-21swapper/323:30:193
1864999374,23cyclictest0-21swapper/323:30:193
18649993718,4cyclictest13883-21sh23:45:113
1864999371,6cyclictest0-21swapper/320:25:193
1864999371,4cyclictest7750irq/56-dwc_otg22:40:183
1864999371,4cyclictest7750irq/56-dwc_otg22:40:183
1864899376,5cyclictest0-21swapper/223:35:062
1864899376,5cyclictest0-21swapper/223:35:062
18648993720,4cyclictest840-21sh22:43:482
18648993720,4cyclictest840-21sh22:43:482
18648993715,4cyclictest6019-21if_eth022:50:262
18648993713,4cyclictest31499-21cut23:25:192
18648993712,4cyclictest301ktimersoftd/222:14:042
18647993721,3cyclictest7750irq/56-dwc_otg23:50:181
18647993715,5cyclictest1475050wget00:30:211
18647993710,4cyclictest0-21swapper/100:22:441
18649993613,5cyclictest1120-21runrttasks00:13:473
1864999361,25cyclictest29614-21diskmemload23:25:063
18649993612,5cyclictest18746-21/usr/sbin/munin00:35:263
1864899367,5cyclictest0-21swapper/221:25:052
1864899367,21cyclictest0-21swapper/221:13:582
1864899367,21cyclictest0-21swapper/221:13:582
18648993617,5cyclictest24621-21sh22:30:492
18648993611,4cyclictest0-21swapper/223:17:162
1864799362,5cyclictest0-21swapper/120:15:051
18647993625,8cyclictest0-21swapper/100:25:231
1864799362,19cyclictest0-21swapper/123:10:111
18647993616,5cyclictest32089-21missed_timers23:25:281
18647993615,5cyclictest22198-21latency_hist20:50:061
1864799361,17cyclictest26748-21ssh22:35:061
1864999352,19cyclictest22961-21ls20:50:273
1864999352,19cyclictest22961-21ls20:50:273
18649993514,12cyclictest140-21systemd-journal20:05:403
18649993514,12cyclictest140-21systemd-journal20:05:403
1864899352,5cyclictest12333-21ssh21:30:192
18648993517,11cyclictest3768-21ssh21:17:502
18648993516,4cyclictest1120-21runrttasks22:20:472
1864799358,5cyclictest7750irq/56-dwc_otg23:35:061
1864799358,5cyclictest7750irq/56-dwc_otg23:35:061
1864799358,5cyclictest23-21ksoftirqd/122:50:371
1864799353,30cyclictest7950irq/56-dwc_otg_21:45:181
1864799353,19cyclictest7950irq/56-dwc_otg_22:20:341
1864799352,5cyclictest7950irq/56-dwc_otg_00:10:201
18647993515,6cyclictest13079-21cpu23:00:201
18647993515,6cyclictest13079-21cpu23:00:201
18649993416,6cyclictest1120-21runrttasks00:20:313
18649993416,4cyclictest23837-21sh22:30:273
1864999341,22cyclictest9819-21ls22:10:413
1864999341,22cyclictest30109-21cat21:55:063
1864999341,21cyclictest0-21swapper/320:32:533
1864999341,13cyclictest0-21swapper/323:18:473
1864899344,15cyclictest7750irq/56-dwc_otg19:40:162
18648993430,2cyclictest0-21swapper/200:35:252
18648993417,5cyclictest15146-21ssh23:45:342
18648993416,4cyclictest11270-21sh22:57:092
18648993415,12cyclictest301ktimersoftd/222:05:062
18648993415,12cyclictest301ktimersoftd/222:05:062
18648993412,4cyclictest32167-21seq21:56:562
1864899341,12cyclictest22699-21ssh23:55:332
18648993410,14cyclictest0-21swapper/223:51:152
1864799348,6cyclictest7950irq/56-dwc_otg_20:40:051
1864799345,6cyclictest7750irq/56-dwc_otg21:20:361
1864799343,4cyclictest13715-21fschecks_time20:25:251
1864799342,5cyclictest18516-21diskstats00:35:241
18647993419,8cyclictest24909-21ssh22:31:261
18647993413,5cyclictest14842-21sh21:33:221
1864799341,13cyclictest7850irq/56-dwc_otg_20:15:251
115882341,29sleep211597-21ssh21:29:132
5722330,5sleep21864899cyclictest00:10:232
18649993315,11cyclictest13319-21cat20:25:063
18649993312,5cyclictest17791-21cat23:05:383
1864899338,4cyclictest0-21swapper/200:25:222
1864899335,25cyclictest0-21swapper/219:20:182
1864899334,5cyclictest7750irq/56-dwc_otg22:37:112
18648993320,7cyclictest7950irq/56-dwc_otg_21:54:392
1864899331,23cyclictest7749-21latency_hist20:10:062
1864899331,23cyclictest7749-21latency_hist20:10:062
1864799339,13cyclictest0-21swapper/123:05:271
1864799334,11cyclictest7850irq/56-dwc_otg_21:50:201
1864799333,16cyclictest2627-21proc_pri22:45:321
1864799332,12cyclictest7850irq/56-dwc_otg_20:05:061
18647993319,5cyclictest20223-21cron19:15:051
18647993315,11cyclictest29614-21diskmemload22:04:381
172752336,15sleep30-21swapper/319:05:263
170822338,10sleep10-21swapper/119:05:241
288412320,2sleep339-21ksoftirqd/322:37:013
285982321,2sleep30-21swapper/300:04:383
1864999326,17cyclictest0-21swapper/323:35:313
1864999325,11cyclictest7850irq/56-dwc_otg_20:55:413
18649993222,7cyclictest23626-21if_wlan021:45:263
18649993220,5cyclictest7950irq/56-dwc_otg_21:55:233
18649993215,9cyclictest10510-21sh20:15:313
1864999321,4cyclictest7750irq/56-dwc_otg23:26:203
1864999321,17cyclictest27893-21gpgconf21:05:063
18649993210,7cyclictest381ktimersoftd/321:20:363
1864899327,23cyclictest0-21swapper/221:00:202
1864899323,9cyclictest0-21swapper/200:30:322
1864899322,20cyclictest0-21swapper/219:46:502
1864899321,26cyclictest1946-21ssh22:45:212
1864899321,24cyclictest23356-21ls21:45:222
18648993212,11cyclictest12698-21fschecks_count22:15:242
18648993211,10cyclictest0-21swapper/222:00:192
1864799321,7cyclictest3723-21grep19:55:341
1864799321,5cyclictest23-21ksoftirqd/122:05:231
1864799321,5cyclictest23-21ksoftirqd/122:05:231
1864999319,4cyclictest0-21swapper/322:00:483
1864999317,10cyclictest7950irq/56-dwc_otg_21:05:293
1864999316,11cyclictest7950irq/56-dwc_otg_21:15:063
1864999316,11cyclictest7950irq/56-dwc_otg_21:15:063
1864999311,3cyclictest0-21swapper/320:40:263
18649993112,4cyclictest17947-21latency_hist00:35:063
18649993112,12cyclictest4386-21/usr/sbin/munin20:00:183
1864899319,16cyclictest0-21swapper/221:43:152
1864899319,16cyclictest0-21swapper/221:43:152
1864899317,3cyclictest0-21swapper/223:22:252
1864899313,13cyclictest30064-21ssh00:05:312
1864899311,25cyclictest7950irq/56-dwc_otg_20:30:252
1864899311,18cyclictest0-21swapper/221:38:252
1864799319,5cyclictest0-21swapper/121:29:101
1864799318,5cyclictest7750irq/56-dwc_otg20:05:221
1864799318,5cyclictest7750irq/56-dwc_otg20:05:221
1864799313,7cyclictest20891-21fschecks_time19:15:241
1864799313,7cyclictest20891-21fschecks_time19:15:241
1864799312,5cyclictest7950irq/56-dwc_otg_23:15:181
1864799312,3cyclictest0-21swapper/119:25:171
18647993117,5cyclictest1442-21/usr/sbin/munin19:50:271
18647993113,9cyclictest29027-21timerwakeupswit21:05:331
1864999309,5cyclictest0-21swapper/322:55:303
1864999308,13cyclictest13587-21missed_timers23:00:273
1864999308,13cyclictest13587-21missed_timers23:00:273
1864999307,13cyclictest1120-21runrttasks00:08:493
1864999307,13cyclictest0-21swapper/321:35:203
1864999306,14cyclictest14516-21ssh21:32:383
1864999301,5cyclictest7750irq/56-dwc_otg19:10:193
1864899306,4cyclictest7850irq/56-dwc_otg_19:35:352
1864899302,16cyclictest4706-21ssh00:15:322
1864899301,9cyclictest20358-21chmod19:15:062
1864799305,12cyclictest7850irq/56-dwc_otg_23:21:191
1864799303,13cyclictest13198-21ssh22:15:311
18647993016,4cyclictest21-21rcuc/119:30:341
1864999295,13cyclictest0-21swapper/322:15:113
1864999291,4cyclictest0-21swapper/322:22:293
1864999291,22cyclictest30476-21ls19:40:283
1864999291,19cyclictest0-21swapper/319:53:503
1864999291,19cyclictest0-21swapper/319:39:153
1864999291,17cyclictest0-21swapper/300:25:183
1864999291,12cyclictest0-21swapper/323:53:293
1864999291,10cyclictest2175350ssh21:42:433
1864999291,10cyclictest2175350ssh21:42:433
1864899296,13cyclictest224232chrt20:50:162
1864899296,13cyclictest224232chrt20:50:162
1864799297,19cyclictest0-21swapper/120:40:221
1864799294,5cyclictest7750irq/56-dwc_otg19:20:321
1864799290,20cyclictest2026-21ssh21:15:271
1864999281,9cyclictest0-21swapper/320:10:373
1864899281,16cyclictest0-21swapper/220:15:222
1864799282,6cyclictest0-21swapper/121:55:211
1864999271,14cyclictest0-21swapper/319:22:093
1864899271,6cyclictest1160450/usr/sbin/munin20:20:242
1864899271,11cyclictest7750irq/56-dwc_otg20:28:042
1864799277,11cyclictest7850irq/56-dwc_otg_20:30:381
1864799271,4cyclictest7750irq/56-dwc_otg19:35:271
1864799271,12cyclictest7750irq/56-dwc_otg20:20:201
18648992620,4cyclictest5977-21latency_hist20:05:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional