You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 17:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Aug 23, 2026 12:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8721991453,11cyclictest29636-21grep09:20:380
8721991343,9cyclictest12562-21apt-get09:40:220
8721991343,9cyclictest12562-21apt-get09:40:220
8722991303,10cyclictest11136-21/usr/sbin/munin10:55:401
8722991265,12cyclictest29597-21/usr/sbin/munin09:20:371
8721991194,92cyclictest0-21swapper/012:10:390
8722991185,90cyclictest12130-21ssh12:15:351
8721991184,13cyclictest7850irq/56-dwc_otg_10:55:390
2429521150,13sleep024164-21/usr/sbin/munin09:15:260
8721991142,31cyclictest9900-21ssh10:15:350
8721991118,94cyclictest13556-21kworker/0:110:05:390
8722991067,77cyclictest1180-21runrttasks10:35:281
8722991066,74cyclictest7950irq/56-dwc_otg_09:40:231
8722991066,74cyclictest7950irq/56-dwc_otg_09:40:231
87239910437,40cyclictest29736-21/usr/sbin/munin09:20:402
87219910223,11cyclictest12199-21uname12:15:360
8721991012,31cyclictest21990-21fschecks_time10:30:300
2325621010,2sleep00-21swapper/011:10:370
872199995,31cyclictest26272-21ssh10:35:310
872199993,9cyclictest32527-21apt-get09:25:230
8721999923,70cyclictest25948-1kworker/u9:211:20:360
8721999910,6cyclictest0-21swapper/012:35:370
8721999910,6cyclictest0-21swapper/012:35:370
8724999818,61cyclictest0-21swapper/309:40:243
8724999818,61cyclictest0-21swapper/309:40:243
872199982,27cyclictest0-21swapper/011:25:270
872299975,10cyclictest19602sleep110:05:401
872199973,35cyclictest24358-21tail11:50:390
872299962,69cyclictest24282-21sh11:50:381
8721999621,11cyclictest20115-21ssh11:45:390
872299959,60cyclictest8350-21apt-get09:35:221
109622950,12sleep2872399cyclictest10:55:372
8723999436,40cyclictest16242-21snmpd12:15:352
872299936,80cyclictest20262-21grep11:45:411
872299933,68cyclictest21984-21ssh10:30:301
872199934,66cyclictest30252-21tune2fs10:40:290
8724999136,17cyclictest29733-21users09:20:393
872199913,7cyclictest0-21swapper/009:35:220
8723999022,44cyclictest26193-21/usr/sbin/munin10:35:292
8723998930,43cyclictest12562-21apt-get09:40:212
8723998930,43cyclictest12562-21apt-get09:40:212
872299892,10cyclictest9987-21expr10:15:361
872399853,51cyclictest0-21swapper/211:50:372
8723998528,31cyclictest22556-21ssh10:30:402
8723998524,19cyclictest2001-21/usr/sbin/munin10:05:412
8723998521,24cyclictest2637-21ssh11:25:272
872199859,22cyclictest10661-21ssh11:35:230
8724998430,22cyclictest2000-21ssh10:05:413
872299834,72cyclictest28994-21grep12:35:371
872299834,72cyclictest28994-21grep12:35:371
872299833,10cyclictest10638-21apt-get11:35:241
872299824,71cyclictest30148-21sh10:40:271
872199823,10cyclictest17090-21apt-get10:25:110
872299813,71cyclictest1180-21runrttasks09:25:221
872299813,10cyclictest8483-21ssh12:10:401
872299803,55cyclictest0-21swapper/111:20:351
8724997923,40cyclictest11162-21ssh10:55:423
872299784,7cyclictest0-21swapper/111:25:271
8721997817,6cyclictest7850irq/56-dwc_otg_07:55:360
8723997719,21cyclictest0-21swapper/211:20:362
872199751,24cyclictest23898-21latency_hist09:15:090
872499742,19cyclictest12100-21sh12:15:353
8724997413,38cyclictest0-21swapper/310:15:363
872199732,24cyclictest27636-21grep11:15:410
256352730,6sleep3872499cyclictest09:55:313
872499728,46cyclictest7750irq/56-dwc_otg10:40:283
8724997226,18cyclictest22042-21perl10:30:313
872399726,24cyclictest30249-21grep10:40:292
8723997219,26cyclictest8365-21ssh09:35:232
872499716,38cyclictest0-21swapper/311:25:263
8724997126,23cyclictest8472-21users12:10:393
872299713,61cyclictest0-21swapper/110:25:141
872399703,11cyclictest29009-21unixbench-2d12:35:372
872399703,11cyclictest29009-21unixbench-2d12:35:372
872199691,24cyclictest10797-21cut08:45:240
872399682,46cyclictest10126-21ls10:15:392
872199682,5cyclictest0-21swapper/011:07:010
872399675,19cyclictest0-21swapper/211:45:422
872299674,51cyclictest28547-21vmstat11:55:381
872499667,40cyclictest0-21swapper/309:35:223
872499664,11cyclictest31512-21/usr/sbin/munin11:20:353
872499663,38cyclictest0-21swapper/312:35:363
872499663,38cyclictest0-21swapper/312:35:363
8723996623,22cyclictest32542-21ssh09:25:212
8724996523,25cyclictest26217-21chrt10:35:293
872399658,19cyclictest7750irq/56-dwc_otg10:25:112
8724996432,12cyclictest24311-21unixbench-2d11:50:383
8724996424,26cyclictest7850irq/56-dwc_otg_11:45:443
872399648,10cyclictest8463-21uptime12:10:392
872299631,52cyclictest5642-21taskset12:08:001
73422630,2sleep30-21swapper/309:33:573
872399619,8cyclictest0-21swapper/211:35:212
872199612,17cyclictest12250-21latency_hist07:20:100
872299606,45cyclictest7750irq/56-dwc_otg11:15:391
872299602,50cyclictest0-21swapper/111:30:261
872299601,12cyclictest7850irq/56-dwc_otg_09:15:341
872199602,32cyclictest30339-21unixbench_multi10:00:390
872299595,5cyclictest17088-1kworker/1:2H07:35:161
872299594,7cyclictest17337-21ssh09:45:321
872299592,12cyclictest0-21swapper/108:40:271
872299591,5cyclictest29267-21cpu10:00:231
872299591,13cyclictest0-21swapper/107:55:311
872199597,37cyclictest0-21swapper/010:45:290
872199596,19cyclictest0-21swapper/007:10:230
8721995933,15cyclictest7850irq/56-dwc_otg_12:00:390
872199592,6cyclictest28906-21latency_hist10:00:100
8721995921,16cyclictest13951-21idleruntime-cro07:25:090
872299586,43cyclictest0-21swapper/112:26:391
872299583,12cyclictest0-21swapper/112:04:201
872199582,7cyclictest13594-21fschecks_count10:20:270
872199582,7cyclictest13594-21fschecks_count10:20:270
872499574,26cyclictest32549-21runrttasks09:25:223
8724995712,30cyclictest7750irq/56-dwc_otg11:35:233
872299572,8cyclictest9550irq/80-mmc011:40:211
8721995734,5cyclictest17007-21fschecks_count09:45:280
872199572,32cyclictest0-21swapper/011:40:250
8721995716,21cyclictest9450irq/48-DMA08:56:170
872299568,39cyclictest7750irq/56-dwc_otg10:20:281
872299568,39cyclictest7750irq/56-dwc_otg10:20:281
872299562,3cyclictest0-21swapper/109:10:271
872299561,5cyclictest27230-21fschecks_time08:00:271
8721995632,5cyclictest7950irq/56-dwc_otg_08:40:380
872199562,30cyclictest985-21grep08:15:390
8721995617,23cyclictest27483-21idleruntime-cro12:35:090
872299557,4cyclictest28764-21sh10:00:091
872299554,42cyclictest0-21swapper/109:10:101
872299553,45cyclictest0-21swapper/111:04:321
8722995514,4cyclictest9042-21fschecks_time07:10:261
8723995411,22cyclictest25805-21ssh12:32:062
872299545,40cyclictest7850irq/56-dwc_otg_08:50:191
872299543,5cyclictest17648-21sshd09:04:201
872299542,11cyclictest7850irq/56-dwc_otg_08:35:091
872299541,4cyclictest7750irq/56-dwc_otg10:46:441
872299541,4cyclictest0-21swapper/107:25:301
8721995414,28cyclictest0-21swapper/009:34:300
176272540,1sleep20-21swapper/211:04:592
872499533,31cyclictest7750irq/56-dwc_otg10:25:143
872499531,8cyclictest7950irq/56-dwc_otg_07:25:213
872299534,5cyclictest31517-21systemd-journal09:54:031
872299532,3cyclictest0-21swapper/112:30:391
8722995315,35cyclictest30527-21/usr/sbin/munin08:10:261
8722995315,35cyclictest30527-21/usr/sbin/munin08:10:261
872299531,12cyclictest0-21swapper/110:53:181
872199534,5cyclictest0-21swapper/011:30:350
8721995325,12cyclictest0-21swapper/007:35:200
872299523,45cyclictest7850irq/56-dwc_otg_12:20:291
872299522,4cyclictest7950irq/56-dwc_otg_11:10:131
872299521,5cyclictest11196-21ls07:15:321
872299521,40cyclictest0-21swapper/107:47:231
8721995224,4cyclictest1999-21fschecks_time08:20:270
872199522,11cyclictest0-21swapper/012:30:090
8721995220,15cyclictest7950irq/56-dwc_otg_07:30:400
872199521,8cyclictest18249-21grep09:05:280
872199521,7cyclictest3044-21/usr/sbin/munin12:05:370
872499517,28cyclictest0-21swapper/307:15:243
872299514,7cyclictest7950irq/56-dwc_otg_10:10:231
872299513,4cyclictest7750irq/56-dwc_otg08:20:281
872299513,10cyclictest0-21swapper/107:50:231
872299511,5cyclictest10772-21aten_rbpower_cu08:45:241
872299511,5cyclictest0-21swapper/109:34:431
872299511,4cyclictest0-21swapper/108:55:211
872299511,4cyclictest0-21swapper/108:55:211
872299511,42cyclictest0-21swapper/108:37:451
872299511,3cyclictest0-21swapper/107:31:571
872299502,8cyclictest7750irq/56-dwc_otg11:06:201
872299502,10cyclictest0-21swapper/108:08:201
872299501,9cyclictest1013-21sed08:15:391
872299501,8cyclictest21273-21latency_hist07:45:091
8721995027,6cyclictest7950irq/56-dwc_otg_08:30:320
8721995024,5cyclictest7850irq/56-dwc_otg_11:55:350
8721995010,21cyclictest14282-21fschecks_time11:00:270
872399493,11cyclictest22227-21ssh11:10:232
8721994925,6cyclictest7950irq/56-dwc_otg_08:15:090
8721994925,6cyclictest7950irq/56-dwc_otg_08:15:090
872199491,30cyclictest5940-21rm10:10:350
83252493,12sleep00-21swapper/007:08:310
872499487,27cyclictest0-21swapper/307:30:243
872399486,23cyclictest19589-21ssh12:25:232
872199481,26cyclictest23986-21ntpq07:50:320
872399479,25cyclictest29991-21ssh11:58:172
872399479,11cyclictest29554-21expr08:05:362
872399477,25cyclictest0-21swapper/207:35:152
8723994723,17cyclictest608-21ssh12:01:212
8721994724,5cyclictest21590-21ntp_kernel_pll_09:50:320
872499467,25cyclictest0-21swapper/311:40:153
8724994614,15cyclictest7950irq/56-dwc_otg_09:51:553
872399468,21cyclictest7950irq/56-dwc_otg_11:20:092
872399467,24cyclictest0-21swapper/207:30:242
872399467,24cyclictest0-21swapper/207:15:242
872399466,25cyclictest0-21swapper/208:20:232
872399463,8cyclictest7850irq/56-dwc_otg_10:20:222
872399463,8cyclictest7850irq/56-dwc_otg_10:20:222
872399462,23cyclictest20381-21if_wlan009:10:292
8723994613,17cyclictest8558-21ssh10:14:552
8723994611,16cyclictest22599-21ssh09:51:272
872199462,6cyclictest15441-21unixbench_multi08:55:380
872199462,6cyclictest15441-21unixbench_multi08:55:380
8721994622,10cyclictest7950irq/56-dwc_otg_07:28:330
8721994620,5cyclictest6541-21ssh10:50:350
8721994615,4cyclictest4378-21switchtime08:25:340
872199461,25cyclictest0-21swapper/009:00:340
872499458,25cyclictest0-21swapper/307:35:153
872399451,30cyclictest7850irq/56-dwc_otg_09:55:272
872299451,10cyclictest3509-21/usr/sbin/munin08:25:271
872199456,23cyclictest27249-21cut08:00:270
8721994518,7cyclictest8733-21latency_hist08:40:090
872499449,15cyclictest7950irq/56-dwc_otg_11:04:253
872499448,22cyclictest23491-21ssh09:14:373
8724994414,15cyclictest7950irq/56-dwc_otg_12:28:593
872499441,26cyclictest8254-21ssh10:52:553
872399447,23cyclictest0-21swapper/208:40:232
872399442,4cyclictest7950irq/56-dwc_otg_08:35:302
8723994415,12cyclictest7950irq/56-dwc_otg_12:20:392
8723994410,8cyclictest7850irq/56-dwc_otg_11:30:202
872299443,3cyclictest0-21swapper/107:21:311
872199441,26cyclictest20591-21wc07:40:340
872499437,25cyclictest0-21swapper/308:40:233
872499437,24cyclictest0-21swapper/308:05:243
8724994313,16cyclictest7750irq/56-dwc_otg11:20:103
872499422,25cyclictest0-21swapper/308:20:233
872399427,22cyclictest0-21swapper/208:00:232
8723994220,9cyclictest7950irq/56-dwc_otg_09:34:562
8723994214,8cyclictest7850irq/56-dwc_otg_10:00:192
8723994214,5cyclictest5228-21latency_hist08:30:102
872199426,16cyclictest29019-21cut08:05:280
8721994216,16cyclictest0-21swapper/007:45:370
872399419,13cyclictest24884-21ls07:55:092
872399417,11cyclictest11794-21ls08:45:382
8723994114,18cyclictest4875-21ssh12:06:362
8723994113,20cyclictest13618-21unixbench-2d08:50:372
8721994117,6cyclictest7850irq/56-dwc_otg_08:55:090
872499409,15cyclictest7950irq/56-dwc_otg_10:00:203
872499401,22cyclictest7950irq/56-dwc_otg_09:15:323
8724994011,12cyclictest7950irq/56-dwc_otg_11:09:493
872399407,16cyclictest7850irq/56-dwc_otg_11:41:242
187642400,1sleep30-21swapper/312:24:203
872499396,5cyclictest11911-21unixbench_singl08:45:393
872499396,23cyclictest0-21swapper/310:45:223
8724993919,18cyclictest0-21swapper/310:20:143
8724993919,18cyclictest0-21swapper/310:20:143
8724993919,10cyclictest7850irq/56-dwc_otg_12:35:013
8724993911,13cyclictest174282sleep309:01:363
8724993910,15cyclictest381ktimersoftd/307:55:093
872399399,12cyclictest24430-21ssh09:15:282
872399397,11cyclictest1145-21taskset08:16:252
872399396,26cyclictest7950irq/56-dwc_otg_10:46:392
872399396,20cyclictest7950irq/56-dwc_otg_09:45:562
8723993914,14cyclictest301ktimersoftd/211:09:492
872499389,8cyclictest13531-21ls08:50:363
872499388,13cyclictest0-21swapper/308:15:243
8724993815,14cyclictest381ktimersoftd/311:33:083
872499381,21cyclictest7736-21ls08:35:303
8724993811,7cyclictest7750irq/56-dwc_otg11:55:253
8724993811,19cyclictest381ktimersoftd/308:13:423
8724993811,19cyclictest381ktimersoftd/308:13:423
872399387,24cyclictest0-21swapper/208:55:232
872399387,24cyclictest0-21swapper/208:55:232
8723993815,5cyclictest25155-21grep07:55:242
872499374,14cyclictest7950irq/56-dwc_otg_12:09:303
872399377,9cyclictest19560-21ssh09:10:092
872399376,21cyclictest0-21swapper/210:51:042
8723993712,7cyclictest6660-21sshd08:32:202
872499366,13cyclictest7950irq/56-dwc_otg_10:12:033
8723993610,5cyclictest15434-21sshd07:26:202
872499359,4cyclictest27030-21/usr/sbin/munin08:00:243
872499359,15cyclictest381ktimersoftd/307:55:253
872499358,12cyclictest650-21ssh12:01:253
872499352,19cyclictest7950irq/56-dwc_otg_09:48:453
8724993512,11cyclictest5008-21sshd08:29:203
872399359,8cyclictest0-21swapper/208:13:422
872399359,8cyclictest0-21swapper/208:13:422
8724993410,13cyclictest14849-21cat08:55:303
8724993410,13cyclictest14849-21cat08:55:303
872399348,12cyclictest7950irq/56-dwc_otg_07:45:092
8724993313,7cyclictest7750irq/56-dwc_otg11:12:073
8724993310,15cyclictest21780-21fschecks_count07:45:283
8723993311,6cyclictest7950irq/56-dwc_otg_07:10:222
872399326,12cyclictest7950irq/56-dwc_otg_07:20:362
8723993216,4cyclictest301ktimersoftd/207:50:092
872399316,11cyclictest7950irq/56-dwc_otg_09:00:272
872499304,11cyclictest16242-21snmpd08:33:463
872499301,12cyclictest7950irq/56-dwc_otg_09:05:243
872499295,6cyclictest7850irq/56-dwc_otg_07:40:253
872499295,4cyclictest9966-21chrt07:10:513
8724992919,4cyclictest13092-21sh07:20:353
83922274,15sleep30-21swapper/307:09:233
81762274,16sleep10-21swapper/107:06:331
81342263,14sleep20-21swapper/207:06:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional