You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 08:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Jul 02, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1017299926,62cyclictest0-21swapper/023:50:120
1017299826,37cyclictest0-21swapper/021:10:240
1017299811,9cyclictest24467-21ssh22:30:380
10172998016,39cyclictest8318-21ssh22:50:380
133732770,6sleep21018699cyclictest00:15:292
10172997717,45cyclictest0-21swapper/023:50:350
1017299771,61cyclictest25651-21timerandwakeup19:50:370
10172997634,27cyclictest2281-21latency_hist22:05:120
318942750,4sleep21018699cyclictest21:21:402
10172997516,38cyclictest9589-21ntp_states00:10:370
10172997417,46cyclictest9158-21timerandwakeup23:30:380
219462730,5sleep11017999cyclictest23:07:081
1017299738,54cyclictest13920-21basename19:20:230
1017299732,52cyclictest19357-21ssh22:25:270
1017299731,50cyclictest832-21switchtime23:20:390
10172997314,45cyclictest0-21swapper/020:18:090
10172997214,47cyclictest0-21swapper/021:00:340
1017299721,20cyclictest7404-21ssh22:10:350
1017299716,46cyclictest0-21swapper/000:30:280
1017299711,51cyclictest10823-21rm21:35:360
1017299711,33cyclictest3273-21ssh22:45:240
1017299711,14cyclictest22819-21ssh00:26:140
10552700,1sleep30-21swapper/321:24:353
10172997017,41cyclictest14428-21fschecks_count20:50:280
10172997017,28cyclictest25002-21ls23:10:370
1017299695,40cyclictest0-21swapper/023:05:250
1017299694,55cyclictest15058-21ssh23:38:240
1017299691,4cyclictest19621-21ssh23:05:110
68022680,1sleep10-21swapper/122:49:441
1017299681,58cyclictest0-21swapper/020:38:040
1017299681,4cyclictest4320-21idleruntime-cro00:05:110
10172996810,45cyclictest0-21swapper/023:44:520
1017299676,10cyclictest0-21swapper/022:59:130
10172996740,5cyclictest25909-21aten_rbpower_cu21:15:260
10172996615,38cyclictest7850irq/56-dwc_otg21:55:380
10172996612,41cyclictest0-21swapper/023:27:070
10172996612,41cyclictest0-21swapper/023:27:070
1017299661,15cyclictest32485-21ssh21:22:450
10172996611,45cyclictest4532-21sh21:28:230
1017299654,7cyclictest0-21swapper/000:21:580
1017299654,42cyclictest0-21swapper/021:32:290
1017299651,6cyclictest17536-21apt21:00:110
10172996515,40cyclictest7950irq/56-dwc_otg_21:45:400
1017299651,14cyclictest0-21swapper/000:19:580
10172996415,38cyclictest0-21swapper/000:08:060
1017299641,47cyclictest0-21swapper/022:44:420
1017299641,46cyclictest13592-21runrttasks20:46:080
10172996414,39cyclictest29805-21taskset23:16:200
1017299641,42cyclictest0-21swapper/022:39:110
160792630,2sleep316061-21ssh23:40:113
157822630,5sleep07850irq/56-dwc_otg19:25:260
10186996312,20cyclictest8050irq/56-dwc_otg_23:05:242
1017299631,50cyclictest3212-21/usr/sbin/munin22:05:330
10172996310,10cyclictest23955-21ssh21:51:000
72362610,1sleep20-21swapper/200:08:002
302672610,2sleep20-21swapper/220:05:252
1017299613,54cyclictest15816-21ssh22:20:370
10172996115,33cyclictest8050irq/56-dwc_otg_21:43:140
10172996115,33cyclictest8050irq/56-dwc_otg_21:43:140
1017299611,15cyclictest30302-21ntpq00:35:330
1017299611,15cyclictest30302-21ntpq00:35:330
1017299602,44cyclictest0-21swapper/020:21:150
1017299595,41cyclictest0-21swapper/022:15:550
1017299592,12cyclictest0-21swapper/020:25:370
1017299592,12cyclictest0-21swapper/020:25:370
1017299591,46cyclictest0-21swapper/019:45:060
1017299591,15cyclictest21143-21diskmemload23:55:290
1017299591,15cyclictest21143-21diskmemload23:55:290
1017299587,8cyclictest0-21swapper/020:10:080
1017299571,53cyclictest11079-21latency20:40:320
96582563,14sleep30-21swapper/319:07:173
1017299569,39cyclictest0-21swapper/019:15:020
255012550,1sleep30-21swapper/323:11:043
1019399554,8cyclictest0-21swapper/323:45:223
1018699557,19cyclictest7950irq/56-dwc_otg_00:10:262
1017299558,8cyclictest20604-21chrt19:36:530
1017299558,8cyclictest20604-21chrt19:36:530
1017299554,11cyclictest8050irq/56-dwc_otg_20:04:140
10172995514,3cyclictest0-21swapper/019:47:320
1018699546,5cyclictest0-21swapper/222:20:232
205352530,1sleep10-21swapper/121:47:201
10193995310,11cyclictest0-21swapper/321:35:233
1017999537,29cyclictest0-21swapper/120:50:241
1017299532,2cyclictest0-21swapper/021:05:280
1017299531,4cyclictest0-21swapper/020:32:250
1017299531,49cyclictest0-21swapper/019:55:470
1017299531,41cyclictest0-21swapper/020:12:530
97272523,44sleep00-21swapper/019:08:110
1018699521,8cyclictest7950irq/56-dwc_otg_21:25:252
1017299522,7cyclictest18672-21unixbench_singl19:30:380
1018699505,9cyclictest7950irq/56-dwc_otg_21:55:232
1017299501,13cyclictest0-21swapper/019:20:030
10186994927,5cyclictest31-21ksoftirqd/223:50:382
1017999486,25cyclictest7850irq/56-dwc_otg00:20:261
1018699477,24cyclictest0-21swapper/219:35:252
1018699477,24cyclictest0-21swapper/219:35:252
1019399467,24cyclictest0-21swapper/319:45:253
1018699462,9cyclictest0-21swapper/222:05:222
1019399451,2cyclictest0-21swapper/322:15:223
1019399447,24cyclictest0-21swapper/320:50:243
1017999447,24cyclictest0-21swapper/123:50:231
83222437,23sleep10-21swapper/119:05:201
51152430,5sleep31019399cyclictest21:29:263
1019399437,23cyclictest0-21swapper/319:30:233
1019399437,22cyclictest0-21swapper/319:10:243
1018699439,18cyclictest0-21swapper/200:00:212
10186994310,6cyclictest7950irq/56-dwc_otg_21:40:222
10186994310,6cyclictest7950irq/56-dwc_otg_21:40:222
1017999433,3cyclictest0-21swapper/122:30:221
10179994314,11cyclictest21143-21diskmemload21:10:131
89752420,2sleep20-21swapper/223:30:362
80822420,2sleep20-21swapper/222:50:342
1019399427,25cyclictest0-21swapper/322:25:253
1019399424,15cyclictest14179-21ssh21:40:243
1019399424,15cyclictest14179-21ssh21:40:243
10193994219,5cyclictest3166250wget22:40:253
1018699427,23cyclictest0-21swapper/220:10:162
10179994217,4cyclictest24266-21fschecks_count23:10:281
10193994120,5cyclictest4906-21cpuspeed00:05:313
1018699417,24cyclictest0-21swapper/219:10:162
1018699417,23cyclictest0-21swapper/223:35:252
1017999419,13cyclictest21143-21diskmemload22:21:461
1017999417,20cyclictest8050irq/56-dwc_otg_20:00:111
10179994119,15cyclictest3764-21vmstat22:05:391
10193994016,12cyclictest381ktimersoftd/300:20:233
1018699407,23cyclictest0-21swapper/221:00:242
1018699407,23cyclictest0-21swapper/220:30:242
1018699401,4cyclictest0-21swapper/223:15:222
1017999408,3cyclictest8050irq/56-dwc_otg_21:00:241
1018699397,22cyclictest0-21swapper/220:20:232
1018699392,3cyclictest0-21swapper/200:30:222
1018699391,27cyclictest10821-21cut20:40:292
1017999394,4cyclictest8050irq/56-dwc_otg_22:40:221
1017999392,6cyclictest7850irq/56-dwc_otg21:40:241
1017999392,6cyclictest7850irq/56-dwc_otg21:40:241
10179993916,5cyclictest7778-21cron23:30:111
10179993916,5cyclictest7778-21cron23:30:111
1017999391,4cyclictest8050irq/56-dwc_otg_22:15:221
95462380,3sleep337-21rcuc/323:30:533
1019399384,6cyclictest7850irq/56-dwc_otg00:30:113
1019399384,4cyclictest16877-21grep19:25:393
10193993817,4cyclictest25774-21fschecks_time00:30:283
10186993814,6cyclictest24988-21latency_hist00:30:112
10186993814,5cyclictest21144-21/usr/sbin/munin23:45:312
1017999385,3cyclictest7850irq/56-dwc_otg00:25:231
10179993814,16cyclictest0-21swapper/123:35:371
10179993813,4cyclictest2791-21timerandwakeup21:25:381
128202371,3sleep312845-21ssh22:55:393
1019399374,8cyclictest15711-21ssh23:00:253
1018699376,7cyclictest7950irq/56-dwc_otg_19:50:392
1018699373,6cyclictest5229-21unixbench_singl23:25:392
1018699373,6cyclictest5229-21unixbench_singl23:25:392
10186993719,4cyclictest17629-21grep23:40:382
1018699371,23cyclictest904-21latency_hist00:40:122
1018699371,23cyclictest904-21latency_hist00:40:122
1018699371,22cyclictest2266-21ls22:05:122
1017999379,14cyclictest8050irq/56-dwc_otg_22:35:281
10193993613,6cyclictest6397-21latency_hist22:10:123
1019399361,21cyclictest28212-21ls20:00:113
1018699367,18cyclictest31-21ksoftirqd/221:45:282
1018699366,8cyclictest11296-21grep21:35:412
1018699361,14cyclictest24086-21unixbench_singl19:45:392
10179993610,11cyclictest0-21swapper/100:04:351
322232351,5sleep10-21swapper/123:20:211
1019399359,4cyclictest0-21swapper/323:20:273
1019399357,18cyclictest0-21swapper/322:47:013
1019399356,20cyclictest7412-21ssh22:50:253
1018699357,15cyclictest0-21swapper/220:25:172
1018699357,15cyclictest0-21swapper/220:25:172
1018699356,20cyclictest0-21swapper/222:45:242
1018699353,19cyclictest13853-21latency_hist19:20:122
1018699352,8cyclictest1664150ssh23:00:392
10186993516,4cyclictest1089-21unixbench_singl23:20:412
1017999357,5cyclictest8050irq/56-dwc_otg_00:10:111
1017999356,18cyclictest0-21swapper/120:10:171
1017999354,4cyclictest7850irq/56-dwc_otg20:40:391
1017999352,8cyclictest13652-21cron19:20:111
1017999351,5cyclictest0-21swapper/121:50:331
10179993512,4cyclictest0-21swapper/100:19:421
10179993510,5cyclictest0-21swapper/123:30:221
1019399346,8cyclictest17552-21grep23:40:383
1019399343,5cyclictest1652-21fschecks_count20:15:293
1019399342,7cyclictest19879-21aten_rbpower_cu23:05:253
10193993416,4cyclictest18380-21aten_rbpower_en21:45:253
10193993414,10cyclictest10595-21aten_rbpower_en20:40:253
1018699349,18cyclictest0-21swapper/222:45:032
1018699348,14cyclictest22789-21users21:10:392
1018699347,5cyclictest31-21ksoftirqd/222:30:332
1018699342,5cyclictest25718-21find21:15:132
1017999347,4cyclictest0-21swapper/100:10:411
1017999346,5cyclictest0-21swapper/123:00:251
1017999343,5cyclictest24766-21wget19:50:251
1017999343,5cyclictest1706-21fschecks_time20:15:291
1017999342,25cyclictest25932-21ssh00:30:301
1017999341,8cyclictest727-21runrttasks23:40:261
1017999341,8cyclictest32046-21latency_hist23:20:121
1017999341,7cyclictest23019-21aten_rbpower_po19:45:261
10179993417,5cyclictest30478-21if_wlan021:20:281
1017999341,5cyclictest21-21rcuc/122:25:161
10179993415,5cyclictest11396-21sh21:35:461
10179993415,4cyclictest727-21runrttasks21:05:401
1017999341,4cyclictest7950irq/56-dwc_otg_23:45:241
1017999341,4cyclictest0-21swapper/121:32:511
1017999341,3cyclictest7850irq/56-dwc_otg19:35:231
1017999341,3cyclictest7850irq/56-dwc_otg19:35:231
10179993412,8cyclictest0-21swapper/122:59:131
1019399339,5cyclictest7950irq/56-dwc_otg_21:05:243
1019399337,5cyclictest7950irq/56-dwc_otg_19:20:113
1019399337,3cyclictest0-21swapper/321:58:043
1019399336,14cyclictest0-21swapper/320:10:173
1019399335,7cyclictest17052-21unixbench_multi20:55:393
1019399335,5cyclictest19887-21/usr/sbin/munin19:35:333
1019399335,5cyclictest19887-21/usr/sbin/munin19:35:333
1019399332,8cyclictest21454-21/usr/sbin/munin21:10:273
1019399332,4cyclictest8050irq/56-dwc_otg_19:40:223
10193993320,4cyclictest16817-21sh22:21:473
10193993315,7cyclictest125-21systemd-journal21:51:393
1018699339,4cyclictest0-21swapper/221:50:252
1018699335,13cyclictest7950irq/56-dwc_otg_23:55:312
1018699335,13cyclictest7950irq/56-dwc_otg_23:55:312
1018699332,6cyclictest0-21swapper/219:30:142
1018699332,5cyclictest31-21ksoftirqd/222:25:142
1018699331,24cyclictest27688-21uniq22:35:282
1018699331,21cyclictest24159-21ls23:10:252
10186993310,5cyclictest31-21ksoftirqd/221:05:352
1017999337,13cyclictest0-21swapper/121:17:221
1017999332,4cyclictest10965-21ntp_kernel_pll_19:10:331
1017999332,4cyclictest0-21swapper/120:40:111
1017999332,24cyclictest0-21swapper/123:55:371
1017999332,24cyclictest0-21swapper/123:55:371
1017999331,4cyclictest7850irq/56-dwc_otg20:00:251
10179993310,8cyclictest0-21swapper/122:10:271
10179993310,12cyclictest0-21swapper/122:50:521
1019399327,15cyclictest0-21swapper/323:50:233
1019399326,6cyclictest7950irq/56-dwc_otg_00:35:403
1019399326,6cyclictest7950irq/56-dwc_otg_00:35:403
10193993217,5cyclictest301-21cat00:00:123
10193993217,5cyclictest301-21cat00:00:123
1019399321,6cyclictest6826-21diskstats22:10:263
10193993216,5cyclictest9139-21fschecks_time00:10:303
10193993212,4cyclictest21143-21diskmemload21:15:323
1019399321,23cyclictest31091-21uniq22:00:273
1019399321,22cyclictest4178-21cpuspeed23:25:253
1019399321,22cyclictest4178-21cpuspeed23:25:253
10193993211,11cyclictest2518250ssh22:31:463
1019399320,21cyclictest7850irq/56-dwc_otg00:00:283
10186993215,8cyclictest21143-21diskmemload22:57:332
10186993215,4cyclictest31-21ksoftirqd/219:55:312
1017999326,16cyclictest0-21swapper/119:30:231
1017999324,4cyclictest0-21swapper/120:30:311
1017999323,20cyclictest7950irq/56-dwc_otg_21:55:281
1017999322,6cyclictest0-21swapper/120:25:291
1017999322,6cyclictest0-21swapper/120:25:291
1017999321,6cyclictest16861-21grep19:25:391
10179993216,7cyclictest7850irq/56-dwc_otg22:00:351
1017999321,4cyclictest8050irq/56-dwc_otg_20:20:221
1017999321,4cyclictest29558-21ssh00:35:231
1017999321,4cyclictest29558-21ssh00:35:231
10179993210,5cyclictest21409-21fschecks_time19:40:281
1019399319,5cyclictest31810-21apt-config20:10:123
1019399316,5cyclictest7950irq/56-dwc_otg_21:00:223
1019399315,5cyclictest7950irq/56-dwc_otg_22:35:283
10193993111,6cyclictest3511-21cut20:20:273
1018699317,11cyclictest7950irq/56-dwc_otg_20:36:232
1018699314,13cyclictest8050irq/56-dwc_otg_19:40:312
1018699312,13cyclictest8050irq/56-dwc_otg_21:34:222
1018699311,4cyclictest0-21swapper/200:24:322
10186993114,6cyclictest13503-21grep20:45:402
10186993110,6cyclictest6552-21/usr/sbin/munin22:10:212
140282300,5sleep21018699cyclictest19:20:242
1019399304,6cyclictest7950irq/56-dwc_otg_21:30:463
1019399301,4cyclictest7950irq/56-dwc_otg_19:20:323
10193993012,5cyclictest13931-21threads00:15:373
1018699307,20cyclictest0-21swapper/220:00:252
1018699307,14cyclictest0-21swapper/222:15:242
1017999302,5cyclictest518-21rs:main1
1019399295,4cyclictest7950irq/56-dwc_otg_23:15:263
1019399295,12cyclictest8050irq/56-dwc_otg_20:25:323
1019399295,12cyclictest8050irq/56-dwc_otg_20:25:323
1019399291,13cyclictest8050irq/56-dwc_otg_20:35:393
10193992911,3cyclictest381ktimersoftd/320:45:273
1018699291,5cyclictest16390-21/usr/sbin/munin20:55:302
1017999294,13cyclictest8050irq/56-dwc_otg_20:55:381
1017999291,4cyclictest7850irq/56-dwc_otg19:24:331
1019399285,4cyclictest7560-21missed_timers20:30:323
1019399281,12cyclictest8050irq/56-dwc_otg_20:03:483
1018699284,13cyclictest8050irq/56-dwc_otg_20:15:292
10186992810,11cyclictest29-21rcuc/219:28:262
92512277,11sleep20-21swapper/219:05:382
1019399276,4cyclictest7950irq/56-dwc_otg_19:50:253
1017999271,4cyclictest0-21swapper/120:05:241
1018699265,4cyclictest7850irq/56-dwc_otg20:50:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional