You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 04:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Mar 17, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2947621690,5sleep21011699cyclictest22:35:242
2947621690,5sleep21011699cyclictest22:35:242
10114991121,79cyclictest10287-21rm21:35:170
101149910773,17cyclictest0-21swapper/000:17:110
26722900,2sleep00-21swapper/021:25:290
304832860,2sleep10-21swapper/121:58:471
170902820,1sleep10-21swapper/123:38:111
261392810,5sleep21011699cyclictest00:28:252
10115997816,32cyclictest7950irq/56-dwc_otg_22:45:151
1011499784,54cyclictest0-21swapper/022:05:150
40932760,5sleep11011599cyclictest21:27:191
10114997511,8cyclictest7638-21ssh22:10:150
10114997435,29cyclictest0-21swapper/020:50:180
1011499741,58cyclictest17578-21latency_hist21:00:050
1011499732,44cyclictest11918-21chrt22:15:170
182172720,1sleep20-21swapper/200:19:032
1011499727,31cyclictest0-21swapper/019:35:170
10114997237,7cyclictest7850irq/56-dwc_otg_00:25:180
1011499716,55cyclictest27601-21unixbench_multi19:55:330
1011499716,55cyclictest27601-21unixbench_multi19:55:330
111702700,2sleep3381ktimersoftd/319:10:313
10114997026,10cyclictest7850irq/56-dwc_otg_20:30:150
1011499702,53cyclictest31485-21unixbench_singl23:15:340
1011499701,57cyclictest7335-21missed_timers00:05:240
1011499698,45cyclictest29069-21aten_rbpower_po22:35:190
1011499698,45cyclictest29069-21aten_rbpower_po22:35:190
278952680,5sleep31011799cyclictest23:11:323
158892680,2sleep30-21swapper/323:36:003
1011499681,56cyclictest25625-21/usr/sbin/munin21:15:290
10114996734,13cyclictest17690-21sh23:00:170
1011499672,54cyclictest0-21swapper/000:10:300
10114996632,18cyclictest0-21swapper/000:35:440
1011499662,51cyclictest22049-21unixbench-2d19:40:320
10114996610,45cyclictest23164-21fschecks_count19:45:220
10114996610,45cyclictest101ktimersoftd/021:20:290
1011599657,7cyclictest9450irq/48-DMA09:12:251
1011499651,51cyclictest22187-21grep23:05:210
10114996431,23cyclictest7750irq/56-dwc_otg22:50:060
1011499642,50cyclictest520-21cron23:15:040
10114996415,44cyclictest101ktimersoftd/020:15:040
1011499641,51cyclictest0-21swapper/022:40:250
1011499641,47cyclictest0-21swapper/019:20:160
10116996312,7cyclictest7850irq/56-dwc_otg_00:10:172
1011499631,52cyclictest0-21swapper/021:30:200
1011499622,48cyclictest3792-21grep20:20:270
10114996224,7cyclictest0-21swapper/020:05:160
143262610,3sleep114328-21ssh00:14:311
1011699619,3cyclictest0-21swapper/200:01:022
1011699619,3cyclictest0-21swapper/200:01:022
10114996130,14cyclictest30901-21aten_rbpower_cu23:55:180
1011499612,48cyclictest17698-21uniq19:30:230
1011499612,48cyclictest17698-21uniq19:30:230
1011499611,25cyclictest22488-21ssh21:10:300
10114996110,27cyclictest101ktimersoftd/023:55:050
98332603,12sleep20-21swapper/219:09:592
1011499602,46cyclictest0-21swapper/020:15:300
10114996019,10cyclictest7850irq/56-dwc_otg_21:55:170
1011599597,17cyclictest0-21swapper/122:32:201
1011499596,38cyclictest20076-21grep21:45:330
1011499596,38cyclictest20076-21grep21:45:330
1011599581,16cyclictest149-21systemd-journal20:20:131
1011499584,9cyclictest0-21swapper/022:55:170
1011499584,9cyclictest0-21swapper/022:55:170
1011499582,8cyclictest26408-21munin-run23:50:050
10114995815,4cyclictest10951-21sh23:30:280
10114995815,4cyclictest10951-21sh23:30:280
1011499581,30cyclictest5021-21aten_rbpower_po20:25:180
1011499573,4cyclictest18119-21ssh23:40:040
1011499572,43cyclictest9625-21df22:50:200
1011499571,26cyclictest6311-21fschecks_count23:25:210
1011499571,18cyclictest9247-21sshd20:35:290
10114995629,15cyclictest0-21swapper/022:24:190
1011499562,25cyclictest12393-21uniq20:45:220
1011499554,36cyclictest0-21swapper/021:04:020
10114995532,12cyclictest7750irq/56-dwc_otg22:25:210
10114995531,11cyclictest908-21runrttasks21:53:350
1011499551,28cyclictest2686-21aten_rbpower_po00:00:190
1011499551,28cyclictest2686-21aten_rbpower_po00:00:190
98212543,12sleep00-21swapper/019:09:490
10114995416,35cyclictest101ktimersoftd/023:23:350
1011499539,7cyclictest20564-21unixbench_multi21:05:320
1011499535,37cyclictest16876-21taskset19:27:060
1011499532,8cyclictest27847-21ssh00:30:240
10114995328,17cyclictest0-21swapper/020:01:460
1011499531,26cyclictest18544-21aten_rbpower_en23:40:190
222982520,2sleep20-21swapper/221:49:222
222982520,2sleep20-21swapper/221:49:222
1011699527,8cyclictest7750irq/56-dwc_otg23:15:182
10114995217,17cyclictest20415-21grep00:20:330
1011499521,12cyclictest0-21swapper/022:35:050
10114995129,12cyclictest0-21swapper/022:04:450
10114995126,12cyclictest16970-21ssh21:42:290
1011799508,14cyclictest0-21swapper/320:50:183
1011699507,7cyclictest0-21swapper/223:25:152
1011699497,28cyclictest0-21swapper/200:05:162
10116994910,7cyclictest7850irq/56-dwc_otg_22:45:152
1011499498,38cyclictest101ktimersoftd/019:15:320
1011799475,5cyclictest7850irq/56-dwc_otg_23:55:153
10117994710,6cyclictest37-21rcuc/300:35:183
10115994710,15cyclictest23-21ksoftirqd/123:25:151
1011499472,22cyclictest0-21swapper/020:40:280
1011799467,25cyclictest0-21swapper/322:00:183
1011799457,24cyclictest0-21swapper/319:45:103
1011799457,23cyclictest0-21swapper/322:40:113
1011799453,27cyclictest7950irq/56-dwc_otg_00:25:173
10114994528,3cyclictest0-21swapper/019:50:480
1011499452,35cyclictest9-21ksoftirqd/019:10:380
1011799447,22cyclictest0-21swapper/323:05:183
1011799447,22cyclictest0-21swapper/320:30:173
1011799446,25cyclictest0-21swapper/320:40:193
1011699445,6cyclictest7850irq/56-dwc_otg_20:15:172
10115994420,8cyclictest17198-21latency_hist19:30:051
1011599439,22cyclictest0-21swapper/119:15:181
1011599431,3cyclictest7950irq/56-dwc_otg_23:50:221
1011599431,14cyclictest7950irq/56-dwc_otg_19:10:211
1011799426,22cyclictest0-21swapper/320:35:183
1011599427,23cyclictest0-21swapper/100:35:121
10115994218,6cyclictest29801-21latency_hist21:20:061
1011599421,20cyclictest31192-21apt-get00:35:051
1011799418,24cyclictest0-21swapper/321:20:183
1011799417,24cyclictest0-21swapper/319:35:193
1011799417,22cyclictest0-21swapper/321:05:183
1011799415,10cyclictest7750irq/56-dwc_otg21:45:163
1011799415,10cyclictest7750irq/56-dwc_otg21:45:163
1011699419,6cyclictest14737-21idleruntime23:35:242
10116994112,5cyclictest25832-21latency_hist23:10:052
1011599417,22cyclictest0-21swapper/122:40:111
202042400,2sleep20-21swapper/223:02:462
1011799402,23cyclictest0-21swapper/320:07:183
10117994018,13cyclictest37-21rcuc/322:30:063
10117993918,11cyclictest37-21rcuc/321:34:303
1011699398,7cyclictest7950irq/56-dwc_otg_22:30:052
1011699395,8cyclictest5748-21latency_hist21:30:052
10115993918,5cyclictest30474-21latency21:20:241
1011799389,17cyclictest37-21rcuc/320:20:313
1011799387,22cyclictest0-21swapper/322:10:183
1011799385,9cyclictest0-21swapper/323:00:173
1011799382,12cyclictest0-21swapper/321:35:183
1011799381,22cyclictest16445-21ls19:25:293
10116993815,7cyclictest5914-21grep20:25:302
1011699381,30cyclictest0-21swapper/222:31:102
1011699381,26cyclictest1975-21rm22:40:342
1011599387,24cyclictest0-21swapper/123:02:321
10115993817,9cyclictest3375-21runrttasks22:05:151
244252370,2sleep10-21swapper/123:07:441
1011799376,21cyclictest0-21swapper/320:15:203
10117993710,10cyclictest7750irq/56-dwc_otg21:42:513
1011699375,6cyclictest27931-21ssh23:50:332
1011699373,22cyclictest0-21swapper/222:24:442
10116993710,5cyclictest6944-21swap21:30:292
1011599378,16cyclictest9450irq/48-DMA09:12:251
1011599377,15cyclictest22545-21cron19:45:041
10115993723,4cyclictest7950irq/56-dwc_otg_21:10:151
1011599371,5cyclictest1680-21ssh22:02:431
90092366,18sleep19091-21/usr/sbin/munin19:05:291
22362360,4sleep32239-21sort23:20:223
1011799362,7cyclictest0-21swapper/319:20:163
1011799362,22cyclictest37-21rcuc/323:15:253
1011799361,24cyclictest13961-21apt-config20:50:053
1011699366,5cyclictest7950irq/56-dwc_otg_19:20:332
1011699365,15cyclictest0-21swapper/223:11:442
1011699362,24cyclictest28456-21sed00:30:322
10116993615,5cyclictest2432-21sh23:20:242
1011699361,21cyclictest24982-21uniq19:50:232
1011599368,15cyclictest23782-21/usr/sbin/munin21:50:261
1011599365,13cyclictest7950irq/56-dwc_otg_20:05:311
1011599364,17cyclictest7850irq/56-dwc_otg_22:55:261
1011599364,17cyclictest7850irq/56-dwc_otg_22:55:261
10115993617,8cyclictest15026-21sh22:18:401
10115993616,5cyclictest1885-21/usr/sbin/munin23:20:221
1011799358,15cyclictest39-21ksoftirqd/319:40:223
1011799356,17cyclictest0-21swapper/300:20:183
10117993521,9cyclictest0-21swapper/300:00:173
10117993521,9cyclictest0-21swapper/300:00:173
10117993514,13cyclictest39-21ksoftirqd/322:55:173
10117993514,13cyclictest39-21ksoftirqd/322:55:173
10117993513,10cyclictest908-21runrttasks23:33:103
10117993513,10cyclictest908-21runrttasks23:33:103
1011799351,15cyclictest7750irq/56-dwc_otg22:32:583
10117993511,11cyclictest7750irq/56-dwc_otg21:15:233
1011699355,5cyclictest0-21swapper/221:20:052
10116993514,5cyclictest7950irq/56-dwc_otg_22:55:282
10116993514,5cyclictest7950irq/56-dwc_otg_22:55:282
10116993514,4cyclictest23544-21if_wlan021:50:232
10116993513,13cyclictest116862chrt20:42:202
1011599358,14cyclictest0-21swapper/100:05:161
1011599356,5cyclictest0-21swapper/120:18:471
1011599356,14cyclictest7950irq/56-dwc_otg_21:35:331
1011599351,24cyclictest19492-21uniq00:20:221
10115993510,14cyclictest0-21swapper/100:19:271
1011799347,15cyclictest39-21ksoftirqd/323:52:303
1011799345,6cyclictest7950irq/56-dwc_otg_23:45:203
1011799345,16cyclictest39-21ksoftirqd/322:21:493
1011799343,10cyclictest39-21ksoftirqd/300:15:043
10117993428,4cyclictest381ktimersoftd/321:00:163
1011799342,7cyclictest37-21rcuc/322:35:253
1011799342,7cyclictest37-21rcuc/322:35:253
1011799341,8cyclictest39-21ksoftirqd/300:15:273
10117993417,8cyclictest29977-21sh00:32:523
1011799341,25cyclictest32262-21ls20:10:233
1011799341,24cyclictest7950irq/56-dwc_otg_21:14:263
1011699349,6cyclictest29-21rcuc/221:55:302
1011699348,14cyclictest0-21swapper/222:15:212
1011699347,6cyclictest7950irq/56-dwc_otg_19:25:212
1011699345,19cyclictest31567-21ssh00:35:122
1011699344,6cyclictest7850irq/56-dwc_otg_22:50:222
10116993414,4cyclictest21881-21sh00:23:112
10116993413,8cyclictest0-21swapper/219:30:172
10116993413,8cyclictest0-21swapper/219:30:172
1011599343,13cyclictest0-21swapper/121:32:331
1011599342,7cyclictest19594-21wget21:05:191
10115993416,8cyclictest686-21cat22:40:061
10115993416,8cyclictest686-21cat22:40:061
1011599341,5cyclictest0-21swapper/123:40:341
10115993414,14cyclictest0-21swapper/100:04:511
10115993414,14cyclictest0-21swapper/100:04:511
1011599341,27cyclictest23561-21uniq00:25:221
10115993410,5cyclictest6052-21unixbench_multi20:25:311
1011799338,10cyclictest37-21rcuc/320:30:053
1011799336,9cyclictest37-21rcuc/323:29:493
1011799333,12cyclictest7750irq/56-dwc_otg22:15:563
10117993316,9cyclictest37-21rcuc/300:06:343
10117993315,11cyclictest7341-21latency_hist22:10:053
1011799331,20cyclictest39-21ksoftirqd/321:59:333
1011699339,5cyclictest7950irq/56-dwc_otg_22:10:302
1011699335,11cyclictest7950irq/56-dwc_otg_20:50:182
1011699333,20cyclictest0-21swapper/221:21:442
1011699331,17cyclictest7950irq/56-dwc_otg_20:05:302
1011599337,15cyclictest0-21swapper/123:59:201
1011599334,11cyclictest765550ssh22:10:171
10115993321,8cyclictest24782-21aten_rbpower_po19:50:201
207542320,3sleep3381ktimersoftd/323:42:203
1011799325,17cyclictest11-21rcu_preempt21:50:483
1011799324,13cyclictest37-21rcuc/319:32:223
1011799324,13cyclictest37-21rcuc/319:32:223
10117993226,4cyclictest24725-21aten_rbpower_cu19:50:193
10117993212,5cyclictest1203350wget19:15:193
10117993212,4cyclictest0-21swapper/320:00:213
1011799321,21cyclictest26784-21grep19:55:233
1011799321,21cyclictest26784-21grep19:55:233
10117993212,10cyclictest37-21rcuc/321:28:413
1011799321,19cyclictest9069-21idleruntime-cro22:50:053
1011699329,4cyclictest7950irq/56-dwc_otg_23:30:182
1011699329,4cyclictest7950irq/56-dwc_otg_23:30:182
1011699328,5cyclictest7850irq/56-dwc_otg_19:45:042
1011699327,4cyclictest0-21swapper/221:12:512
1011699326,6cyclictest2027-21ssh23:59:432
1011699324,13cyclictest7950irq/56-dwc_otg_21:43:472
1011699323,17cyclictest19516-21aten_rbpower_en21:05:182
1011699323,15cyclictest8741-21cpuspeed20:35:202
1011699321,6cyclictest11315-21grep21:35:322
1011699321,5cyclictest19781-21users23:40:352
1011599327,5cyclictest21-21rcuc/123:45:211
1011599327,14cyclictest0-21swapper/120:00:151
1011599326,13cyclictest7750irq/56-dwc_otg21:40:271
1011599324,17cyclictest0-21swapper/123:34:001
1011599324,17cyclictest0-21swapper/123:34:001
1011599323,14cyclictest7850irq/56-dwc_otg_22:50:191
10115993213,7cyclictest21047-21diskmemload21:45:251
10115993213,7cyclictest21047-21diskmemload21:45:251
1011599321,28cyclictest20789-21grep22:25:211
1011599321,19cyclictest28056-21latency_hist20:00:061
1011599321,19cyclictest28056-21latency_hist20:00:061
10115993210,4cyclictest0-21swapper/122:23:161
1011699318,6cyclictest7850irq/56-dwc_otg_22:05:182
1011699317,10cyclictest7950irq/56-dwc_otg_20:00:052
1011699317,10cyclictest7950irq/56-dwc_otg_20:00:052
1011699316,4cyclictest7850irq/56-dwc_otg_20:10:212
1011699316,4cyclictest0-21swapper/222:02:032
1011699314,5cyclictest3646-21latency20:20:252
1011699313,14cyclictest7950irq/56-dwc_otg_20:55:262
1011599318,6cyclictest7950irq/56-dwc_otg_19:35:281
1011599317,5cyclictest7950irq/56-dwc_otg_20:30:261
1011599316,14cyclictest0-21swapper/120:47:321
1011599315,13cyclictest7950irq/56-dwc_otg_20:45:051
1011599312,7cyclictest19036-21latency_hist19:35:061
1011599312,7cyclictest19036-21latency_hist19:35:061
1011599311,5cyclictest17969-21fschecks_time21:00:211
10115993114,11cyclictest15400-21latency_hist19:25:051
1011599311,3cyclictest7850irq/56-dwc_otg_19:45:061
1011799306,14cyclictest0-21swapper/320:55:183
1011699308,4cyclictest0-21swapper/223:45:562
10116993014,4cyclictest6726-21/usr/sbin/munin20:30:242
1011699301,14cyclictest7950irq/56-dwc_otg_19:45:062
1011699301,11cyclictest29470-21users20:00:332
1011599302,9cyclictest0-21swapper/120:11:101
10115993012,5cyclictest14062-21/usr/sbin/munin20:50:271
1011799292,10cyclictest7850irq/56-dwc_otg_22:50:263
1011599291,3cyclictest0-21swapper/120:35:181
1011699285,6cyclictest7950irq/56-dwc_otg_19:35:292
1011699281,6cyclictest12477-21cut20:45:232
1011599281,9cyclictest16191-21fschecks_time20:55:221
96292273,14sleep30-21swapper/319:07:253
1011699276,4cyclictest7850irq/56-dwc_otg_19:10:222
1011699274,5cyclictest7750irq/56-dwc_otg19:15:192
1011699274,4cyclictest17662-21/usr/sbin/munin21:00:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional