You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-15 - 03:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Sep 15, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1677299807,8cyclictest7750irq/56-dwc_otg20:15:321
115492800,2sleep30-21swapper/322:07:273
16772997537,28cyclictest7950irq/56-dwc_otg_21:25:341
1677199741,61cyclictest11888-21fschecks_count20:25:370
8682720,1sleep10-21swapper/123:13:401
8682720,1sleep10-21swapper/123:13:401
291252710,9sleep3381ktimersoftd/300:26:373
1677299716,29cyclictest0-21swapper/122:40:341
1677399707,34cyclictest0-21swapper/222:40:342
16771996928,6cyclictest0-21swapper/020:35:330
16771996914,11cyclictest30043-21ssh22:30:350
155122690,2sleep10-21swapper/122:51:101
1677399677,20cyclictest0-21swapper/221:30:352
1677399677,20cyclictest0-21swapper/221:30:352
16772996611,18cyclictest7750irq/56-dwc_otg21:30:351
16772996611,18cyclictest7750irq/56-dwc_otg21:30:351
1677199664,39cyclictest28880-21unixbench_singl19:40:500
75432650,1sleep00-21swapper/021:24:160
16771996551,10cyclictest7750irq/56-dwc_otg00:00:320
1677199651,46cyclictest20735-21cut19:20:390
16773996411,18cyclictest7850irq/56-dwc_otg_21:25:342
16771996422,5cyclictest5877-21aten_rbpower_en22:40:340
16771996412,5cyclictest0-21swapper/021:00:340
16771996412,37cyclictest24158-21dpkg21:00:210
160792630,2sleep31677499cyclictest22:13:033
160792630,2sleep31677499cyclictest22:13:033
1677299622,10cyclictest7950irq/56-dwc_otg_21:45:321
16771996232,12cyclictest7750irq/56-dwc_otg22:20:380
1677299617,34cyclictest0-21swapper/121:00:261
1677199602,34cyclictest0-21swapper/023:15:290
16771996012,34cyclictest27867-21cron21:10:210
1677199599,34cyclictest18814-21idleruntime-cro00:15:210
1677199599,34cyclictest18814-21idleruntime-cro00:15:210
1677199592,44cyclictest26819-21/usr/sbin/munin22:25:450
1677199591,46cyclictest13158-21http22:10:200
1677199591,10cyclictest22484-21latency_hist20:55:210
1677299586,20cyclictest0-21swapper/121:15:351
16771995836,4cyclictest7750irq/56-dwc_otg23:00:390
16771995813,33cyclictest27573-21grep23:05:490
1677399575,18cyclictest7950irq/56-dwc_otg_20:55:332
16771995710,36cyclictest0-21swapper/022:15:460
324782560,2sleep30-21swapper/321:53:593
1677499562,11cyclictest0-21swapper/300:00:323
1677299566,31cyclictest0-21swapper/121:40:341
1677199568,27cyclictest7378-21unixbench_singl20:10:500
1677199566,46cyclictest0-21swapper/000:30:360
1677199566,28cyclictest30669-21unixbench_singl19:45:480
16771995618,9cyclictest10619-21ssh22:45:410
1677199561,42cyclictest0-21swapper/022:50:480
1677499554,9cyclictest7950irq/56-dwc_otg_21:00:343
1677399556,29cyclictest0-21swapper/219:35:262
1677299559,7cyclictest0-21swapper/123:35:321
1677299551,16cyclictest6494-21grep20:10:391
16771995531,6cyclictest7750irq/56-dwc_otg20:10:210
16771995513,39cyclictest101ktimersoftd/023:40:360
1677199543,20cyclictest0-21swapper/019:55:340
1677199542,7cyclictest297072sleep021:50:390
1677199542,26cyclictest27597-21missed_timers23:45:430
16771995415,31cyclictest6885-21ssh00:40:180
1677199541,43cyclictest0-21swapper/022:01:550
1677199541,43cyclictest0-21swapper/022:01:550
1677199535,41cyclictest0-21swapper/021:13:440
1677199535,28cyclictest32257-21unixbench_multi23:50:490
16771995343,4cyclictest7850irq/56-dwc_otg_00:15:420
16771995333,12cyclictest101ktimersoftd/000:30:020
1677199532,8cyclictest14151-21munin-run23:30:210
16771995326,5cyclictest7750irq/56-dwc_otg23:20:410
1677199531,7cyclictest3929-21munin-run21:20:210
1677199531,7cyclictest25473-21wget21:45:350
1677199531,5cyclictest27863-21diskmemload22:10:480
1677199531,5cyclictest27863-21diskmemload22:10:480
16771995312,34cyclictest0-21swapper/023:30:410
1677299526,15cyclictest31380-21fschecks_time23:50:381
1677299525,4cyclictest0-21swapper/123:45:321
1677199523,28cyclictest0-21swapper/021:40:320
16771995222,17cyclictest0-21swapper/020:20:420
16771995217,7cyclictest1970-21ssh21:55:460
16771995210,34cyclictest16203-21systemd-journal23:37:060
1677399516,7cyclictest7950irq/56-dwc_otg_23:20:352
1677199515,24cyclictest0-21swapper/019:15:440
1677199512,6cyclictest20719-21latency_hist20:50:210
1677199511,30cyclictest1180-21runrttasks19:25:320
1677399506,29cyclictest0-21swapper/220:20:332
1677299504,7cyclictest7750irq/56-dwc_otg00:35:341
1677299504,15cyclictest15038-21rm00:10:341
1677299504,15cyclictest15038-21rm00:10:341
1677199504,4cyclictest27030-21ssh00:25:210
1677199502,28cyclictest25823-21latency_hist19:35:220
1677199501,25cyclictest11210-21fschecks_count00:05:380
1677499491,4cyclictest7950irq/56-dwc_otg_22:45:303
1677399497,27cyclictest32462-21sh21:15:352
16773994911,11cyclictest7950irq/56-dwc_otg_00:35:342
1677299497,27cyclictest0-21swapper/119:35:261
16772994915,19cyclictest13859-21if_eth020:30:401
1677199497,22cyclictest12606-21ls21:30:220
16771994923,12cyclictest8917-21wc20:15:470
1677199491,36cyclictest31324-21/usr/sbin/munin19:50:430
1677199491,36cyclictest31324-21/usr/sbin/munin19:50:430
16773994817,10cyclictest13323-21latency_hist22:10:212
1677299487,25cyclictest0-21swapper/123:00:351
1677299484,7cyclictest0-21swapper/119:45:321
1677199487,20cyclictest0-21swapper/020:45:220
1677199487,20cyclictest0-21swapper/020:45:220
1677199486,4cyclictest17389-21missed_timers19:10:420
16771994821,11cyclictest7950irq/56-dwc_otg_21:40:210
1677399476,24cyclictest0-21swapper/222:35:342
1677399473,5cyclictest0-21swapper/220:15:322
1677299477,27cyclictest0-21swapper/120:35:351
1677299477,25cyclictest0-21swapper/119:40:361
16772994721,9cyclictest7950irq/56-dwc_otg_22:05:261
1677199473,36cyclictest28533-21kworker/0:223:10:410
1677199473,36cyclictest28533-21kworker/0:223:10:410
16771994724,4cyclictest7850irq/56-dwc_otg_21:30:370
16771994724,4cyclictest7850irq/56-dwc_otg_21:30:370
1677199472,26cyclictest3653-21unixbench_singl20:00:500
16771994719,12cyclictest0-21swapper/019:35:230
1677399464,7cyclictest0-21swapper/222:25:312
1677299467,26cyclictest0-21swapper/120:40:371
1677299467,26cyclictest0-21swapper/120:40:371
1677299466,23cyclictest25359-21ssh00:22:151
1677299463,13cyclictest30302-21ssh22:30:391
16772994615,15cyclictest7950irq/56-dwc_otg_20:05:471
1677299461,28cyclictest21067-21ssh22:19:381
1677199463,23cyclictest3037-21ssh22:35:530
1677399457,28cyclictest0-21swapper/220:05:352
16773994525,8cyclictest7950irq/56-dwc_otg_00:30:462
1677299457,25cyclictest0-21swapper/120:20:331
1677299453,6cyclictest0-21swapper/122:25:311
1677299452,8cyclictest7750irq/56-dwc_otg22:00:341
1677299452,8cyclictest7750irq/56-dwc_otg22:00:341
16772994511,13cyclictest3966-21rm23:16:471
1677199458,18cyclictest0-21swapper/022:57:190
16771994514,21cyclictest0-21swapper/020:33:190
161782453,12sleep00-21swapper/019:06:170
44612440,2sleep20-21swapper/223:56:412
1677499446,24cyclictest0-21swapper/319:30:343
1677299448,20cyclictest2787-21ssh00:35:171
1677299446,19cyclictest10056-21modprobe23:25:211
1677299444,19cyclictest32550-21ssh21:54:071
1677199441,14cyclictest5455-21kworker/u8:023:58:430
194092430,5sleep21677399cyclictest19:15:462
1677399437,22cyclictest0-21swapper/221:00:262
16773994310,11cyclictest7950irq/56-dwc_otg_22:25:212
1677299439,19cyclictest2867-21/usr/sbin/munin20:00:401
1677299438,12cyclictest7950irq/56-dwc_otg_20:00:211
1677299436,20cyclictest30279-21ssh21:12:091
16772994320,11cyclictest7950irq/56-dwc_otg_23:30:401
16772994319,8cyclictest7950irq/56-dwc_otg_23:08:151
16772994312,18cyclictest7950irq/56-dwc_otg_22:45:441
178332420,1sleep30-21swapper/323:34:233
1677399427,23cyclictest0-21swapper/200:20:362
16773994222,5cyclictest25636-21fschecks_count21:45:382
16773994212,8cyclictest15022-21apt-get00:10:322
16773994212,8cyclictest15022-21apt-get00:10:322
16773994212,15cyclictest301ktimersoftd/222:30:592
1677299428,12cyclictest25314-21ssh22:24:521
1677299423,25cyclictest4784-21rm22:39:071
16772994218,5cyclictest20238-21wc19:20:211
16772994217,13cyclictest7950irq/56-dwc_otg_22:13:301
16772994217,13cyclictest7950irq/56-dwc_otg_22:13:301
16772994214,13cyclictest7950irq/56-dwc_otg_21:20:321
16772994211,13cyclictest7950irq/56-dwc_otg_20:30:211
16774994110,4cyclictest0-21swapper/322:20:363
16773994123,11cyclictest7850irq/56-dwc_otg_22:13:562
16773994123,11cyclictest7850irq/56-dwc_otg_22:13:562
1677299418,3cyclictest0-21swapper/123:27:511
1677299417,17cyclictest0-21swapper/100:19:261
1677299416,25cyclictest0-21swapper/120:55:331
1677299414,5cyclictest7950irq/56-dwc_otg_21:55:331
16772994111,14cyclictest221ktimersoftd/119:15:221
16774994019,5cyclictest31235-21sh23:50:363
1677399408,4cyclictest15117-21kernelversion23:30:412
1677399405,23cyclictest10926-21ssh00:05:342
1677399403,3cyclictest7950irq/56-dwc_otg_19:40:362
16773994012,14cyclictest7850irq/56-dwc_otg_23:11:052
16773994012,14cyclictest7850irq/56-dwc_otg_23:11:052
16773994010,10cyclictest301ktimersoftd/222:16:452
16772994012,13cyclictest15968-1kworker/u9:222:59:041
1677399397,22cyclictest0-21swapper/223:05:342
1677399397,11cyclictest7850irq/56-dwc_otg_22:00:342
1677399397,11cyclictest7850irq/56-dwc_otg_22:00:342
1677399396,28cyclictest0-21swapper/220:40:262
1677399396,28cyclictest0-21swapper/220:40:262
1677399396,19cyclictest0-21swapper/219:10:342
16773993924,7cyclictest7750irq/56-dwc_otg23:44:552
1677399390,25cyclictest13170-21ssh23:28:342
1677299399,17cyclictest0-21swapper/100:27:331
1677299394,19cyclictest24957-21ssh23:42:231
1677299391,7cyclictest7950irq/56-dwc_otg_21:05:311
16772993917,4cyclictest1421050ssh00:09:291
1677499385,26cyclictest32059-21ls00:30:423
16774993819,5cyclictest31252-21grep22:30:503
16774993813,5cyclictest28821-21tail19:40:503
1677499381,25cyclictest12594-21timerandwakeup20:25:473
1677499381,22cyclictest2704-21ssh23:55:333
1677499381,22cyclictest0-21swapper/323:20:373
1677499380,18cyclictest0-21swapper/300:07:513
1677399386,6cyclictest28591-21unixbench_multi00:25:482
16773993815,10cyclictest301ktimersoftd/223:16:252
1677399381,25cyclictest29824-21grep21:50:392
1677399381,25cyclictest0-21swapper/219:25:352
16773993811,8cyclictest301ktimersoftd/221:10:222
1677299388,5cyclictest6822-21ssh00:00:341
16772993814,7cyclictest221ktimersoftd/123:55:371
1677499374,26cyclictest0-21swapper/320:50:343
1677399377,12cyclictest21515-21ssh22:59:382
1677399376,18cyclictest0-21swapper/220:25:342
1677399373,9cyclictest26914-21sh23:45:322
1677399373,2cyclictest0-21swapper/219:45:332
16773993719,6cyclictest7750irq/56-dwc_otg23:50:372
1677399371,5cyclictest0-21swapper/222:45:322
1677399371,23cyclictest18710-21ssh23:35:342
1677299376,17cyclictest0-21swapper/119:30:341
1677299375,22cyclictest18903-21ssh21:36:521
16774993617,5cyclictest28706-21if_wlan021:10:413
1677499361,23cyclictest20188-21snmpd23:42:073
1677399367,18cyclictest0-21swapper/220:10:352
1677399366,17cyclictest7950irq/56-dwc_otg_19:50:342
1677399366,17cyclictest7950irq/56-dwc_otg_19:50:342
1677399365,19cyclictest21244-21ssh21:40:342
1677399364,21cyclictest301ktimersoftd/220:00:372
1677399363,5cyclictest7750irq/56-dwc_otg00:00:392
1677299362,12cyclictest0-21swapper/119:29:561
16772993611,5cyclictest21374-21missed_timers20:50:411
16774993514,14cyclictest269972chrt21:05:443
1677399353,4cyclictest6790-21ssh21:22:582
16773993516,4cyclictest24180-21/usr/sbin/munin19:30:352
16773993513,4cyclictest7850irq/56-dwc_otg_20:30:392
16773993512,9cyclictest7750irq/56-dwc_otg20:35:472
16773993512,14cyclictest7950irq/56-dwc_otg_21:37:142
243162341,4sleep20-21swapper/223:02:152
1677499344,5cyclictest7850irq/56-dwc_otg_20:00:333
1677499343,22cyclictest0-21swapper/323:15:323
16774993425,6cyclictest24266-21latency_hist21:00:213
16774993420,7cyclictest7750irq/56-dwc_otg23:28:593
16774993411,13cyclictest27192-21latency_hist00:25:213
1677399345,4cyclictest7850irq/56-dwc_otg_20:00:212
1677399344,7cyclictest28590-21cut21:10:402
1677399341,7cyclictest7850irq/56-dwc_otg_22:50:352
16773993410,4cyclictest7950irq/56-dwc_otg_00:16:332
1677299344,5cyclictest21-21rcuc/120:45:431
1677499337,14cyclictest0-21swapper/300:15:493
1677499332,5cyclictest7750irq/56-dwc_otg23:00:393
1677499332,5cyclictest5873-21ssh22:40:343
16774993314,10cyclictest16203-21systemd-journal22:36:553
1677499331,22cyclictest23781-21latency_hist19:30:213
1677399336,20cyclictest0-21swapper/221:55:342
1677499324,14cyclictest0-21swapper/320:06:503
1677499322,7cyclictest7850irq/56-dwc_otg_20:40:273
1677499322,7cyclictest7850irq/56-dwc_otg_20:40:273
16774993222,4cyclictest3005-21sh21:56:513
16774993219,6cyclictest15105-21ssh21:32:393
16774993219,6cyclictest15105-21ssh21:32:393
1677499321,3cyclictest0-21swapper/321:35:543
16774993213,9cyclictest381ktimersoftd/322:25:453
1677499321,25cyclictest13779-21cut20:30:393
1677499321,16cyclictest9760-21ls20:20:213
16774993211,5cyclictest7950irq/56-dwc_otg_19:45:483
16774993211,14cyclictest186462chrt23:35:283
1677499319,3cyclictest0-21swapper/322:04:013
1677499319,3cyclictest0-21swapper/322:04:013
1677499317,12cyclictest7850irq/56-dwc_otg_23:45:443
1677499313,23cyclictest7950irq/56-dwc_otg_21:40:323
1677499311,5cyclictest0-21swapper/320:15:213
1677499311,4cyclictest0-21swapper/322:58:243
16774993113,5cyclictest3927-21modprobe21:20:213
16774993111,5cyclictest27863-21diskmemload21:22:343
1677499311,11cyclictest0-21swapper/300:37:223
1677399317,5cyclictest0-21swapper/220:45:422
1677399316,14cyclictest0-21swapper/219:20:342
1677499305,4cyclictest15271-21grep22:50:493
1677499304,5cyclictest7850irq/56-dwc_otg_19:55:393
1677499304,5cyclictest7750irq/56-dwc_otg23:05:493
1677499303,22cyclictest0-21swapper/319:50:343
1677499303,22cyclictest0-21swapper/319:50:343
16774993023,4cyclictest25387-21aten_rbpower_cu21:45:343
16774993021,6cyclictest30758-21sort23:10:373
16774993021,6cyclictest30758-21sort23:10:373
16774993012,9cyclictest27863-21diskmemload00:13:573
16774993012,9cyclictest27863-21diskmemload00:13:573
1677399301,3cyclictest0-21swapper/220:50:392
1677299306,14cyclictest0-21swapper/119:20:341
1677299301,13cyclictest30856-21kworker/1:019:55:221
1677299301,13cyclictest30856-21kworker/1:019:55:221
1677499298,4cyclictest0-21swapper/321:25:383
1677499292,12cyclictest0-21swapper/322:16:243
16774992912,10cyclictest10387-21cat20:20:403
1677499287,13cyclictest19307-21awk19:15:453
1677499282,4cyclictest16360-21grep20:35:493
1677499281,14cyclictest0-21swapper/319:15:223
16774992719,5cyclictest1180-21runrttasks20:45:403
1677499271,17cyclictest0-21swapper/319:25:213
162212273,15sleep20-21swapper/219:06:522
153742273,12sleep10-21swapper/119:05:411
149612275,11sleep30-21swapper/319:05:353
16774992619,5cyclictest26250-21fschecks_time19:35:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional