You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 08:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Aug 20, 2026 00:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1130621040,1sleep00-21swapper/023:00:270
1240721020,2sleep012408-21/usr/sbin/munin23:40:330
5291991016,63cyclictest0-21swapper/000:30:220
5291991006,61cyclictest0-21swapper/022:55:210
5291991006,61cyclictest0-21swapper/022:55:210
5291991001,55cyclictest0-21swapper/023:50:210
191632930,2sleep00-21swapper/023:10:140
66722880,1sleep20-21swapper/221:35:472
35642850,2sleep13548-21idleruntime-cro20:35:071
529199782,58cyclictest0-21swapper/019:45:080
5292997755,16cyclictest5290-21cyclictest19:10:091
529199764,20cyclictest9-21ksoftirqd/019:35:180
198622760,2sleep20-21swapper/223:50:212
24972750,5sleep3529499cyclictest21:30:403
24972750,5sleep3529499cyclictest21:30:403
5291997445,23cyclictest0-21swapper/020:25:220
529199741,39cyclictest0-21swapper/021:50:200
41102720,1sleep20-21swapper/222:51:072
529199712,53cyclictest1354-21modprobe22:10:070
529199712,53cyclictest0-21swapper/022:25:190
5291997026,12cyclictest0-21swapper/022:20:190
264252700,1sleep10-21swapper/121:20:401
529199683,46cyclictest0-21swapper/021:30:190
529199683,46cyclictest0-21swapper/021:30:190
529199671,34cyclictest0-21swapper/023:05:180
529199671,32cyclictest0-21swapper/020:50:170
119842670,2sleep20-21swapper/222:21:522
5292996528,20cyclictest24164-21open_inodes23:15:311
529499644,54cyclictest1180-21runrttasks19:10:103
529499642,46cyclictest14819-21ssh21:45:353
529499641,53cyclictest16020-21ssh22:26:413
5292996426,22cyclictest32717-21ssh22:46:391
529199649,37cyclictest25275-21grep23:55:360
529499631,54cyclictest8791-21ssh22:18:353
529499631,54cyclictest6881-21ssh21:36:103
529499631,51cyclictest24637-21ssh21:58:283
529199632,46cyclictest26027-21tail00:35:350
529199631,46cyclictest29212-21/usr/sbin/munin00:00:410
529199631,45cyclictest29523-21ssh21:25:230
52252630,3sleep15226-21sh22:14:391
529499621,53cyclictest20822-21ssh22:32:443
529499621,52cyclictest9336-21ssh21:40:153
5292996229,18cyclictest18401-21chrt00:26:361
5292996228,20cyclictest29965-21rm22:44:111
5292996228,18cyclictest2164-21ls21:30:351
5292996228,18cyclictest2164-21ls21:30:351
5291996242,6cyclictest12931-21aten_rbpower_vo21:00:220
5291996226,6cyclictest25104-21ssh21:20:230
529499611,52cyclictest14758-21ssh23:44:023
529499611,47cyclictest25485-21df20:05:223
529499611,43cyclictest29320-21apt-get21:25:203
529499611,43cyclictest23637-21apt-get23:55:143
529499611,42cyclictest6578-21ssh22:55:123
529499611,42cyclictest6578-21ssh22:55:123
5292996128,9cyclictest22130-21apt-get22:35:131
5291996112,19cyclictest9-21ksoftirqd/023:15:180
48252613,14sleep30-21swapper/319:07:343
48252613,14sleep30-21swapper/319:07:343
529499601,45cyclictest3999-21diskstats20:35:233
529499601,43cyclictest24327-21sh21:19:273
529499601,41cyclictest7092-21ssh23:35:053
5292996026,18cyclictest24735-21sh00:35:121
5292996024,22cyclictest1210-21ssh23:26:541
529199601,36cyclictest0-21swapper/022:35:370
529199601,34cyclictest11296-21fschecks_time20:55:250
529199601,34cyclictest11296-21fschecks_time20:55:250
529499591,50cyclictest2047-21ssh22:49:553
529499591,43cyclictest15535-21users19:35:363
5292995922,21cyclictest6158-21ssh23:33:231
529199591,21cyclictest4045-21cut20:35:230
529499581,40cyclictest8973-21munin-run20:50:083
529499581,38cyclictest1191-21snmpd23:07:013
5293995831,22cyclictest5295-21runrttasks19:10:102
5292995827,16cyclictest9413-21ls00:15:341
5292995824,21cyclictest23296-21ssh21:55:591
5292995822,19cyclictest12960-21rm23:40:441
5292995820,24cyclictest24369-21ssh21:19:311
5292995819,10cyclictest60-21kswapd023:55:141
529199586,22cyclictest31517-21systemd-journal22:11:490
529199584,39cyclictest13692-21sed19:30:340
529199582,28cyclictest4455-21fschecks_time00:10:250
529499572,32cyclictest7950irq/56-dwc_otg_00:27:573
529499571,46cyclictest13022-21df21:00:223
529499571,40cyclictest12572-21aten_rbpower_cu19:30:193
5292995727,17cyclictest32231-21ssh22:07:161
5292995725,12cyclictest12821-21/usr/sbin/munin21:00:181
5292995723,20cyclictest31911-21ssh21:27:511
5292995723,17cyclictest21402-21chrt19:54:541
5292995722,14cyclictest7673-21/usr/sbin/munin23:35:221
5292995722,14cyclictest7287-21/usr/sbin/munin22:55:291
5292995722,14cyclictest7287-21/usr/sbin/munin22:55:291
5291995736,14cyclictest0-21swapper/021:35:230
529199572,37cyclictest28222-21grep20:10:360
5294995626,27cyclictest5075-21rm22:52:523
5294995617,36cyclictest1167-21sh23:26:503
529499561,47cyclictest31996-21sshd20:21:463
529499561,44cyclictest7950irq/56-dwc_otg_23:18:313
529499561,44cyclictest6085-21ssh00:11:463
529499561,43cyclictest25437-21ssh22:38:273
529499561,42cyclictest12339-21taskset20:57:153
529499561,42cyclictest12339-21taskset20:57:153
529499561,40cyclictest27591-21ssh22:01:183
529499561,40cyclictest15531-21ls21:05:333
529499561,40cyclictest13847-21rm23:03:513
529499561,38cyclictest29763-21ssh22:43:493
529499561,37cyclictest3316-21ssh00:09:283
529499561,37cyclictest28988-21ssh23:20:553
529499561,37cyclictest28988-21ssh23:20:553
529499561,35cyclictest7750irq/56-dwc_otg20:00:233
529499561,34cyclictest17533-21chrt23:46:323
5292995625,16cyclictest15712-21awk21:05:351
5292995624,16cyclictest0-21swapper/123:12:391
5292995623,21cyclictest30548-21ssh23:23:471
5292995623,21cyclictest30548-21ssh23:23:471
5292995622,21cyclictest4347-21memory20:35:271
5292995621,19cyclictest20631-21ssh22:32:231
5292995620,25cyclictest1753-21ssh00:06:371
5292995619,24cyclictest19298-21ssh23:49:471
529199565,36cyclictest0-21swapper/019:50:350
5291995634,5cyclictest9-21ksoftirqd/022:35:070
5291995616,11cyclictest0-21swapper/020:30:170
529199561,44cyclictest0-21swapper/022:50:280
529199561,3cyclictest9379-21apt-get21:40:210
529499552,40cyclictest0-21swapper/320:30:173
529499551,43cyclictest31517-21systemd-journal19:59:573
529499551,43cyclictest11527-21ssh22:21:033
529499551,40cyclictest7950irq/56-dwc_otg_00:03:543
529499551,35cyclictest22933-21ssh00:32:203
5292995527,11cyclictest22241-21ssh23:52:371
5292995525,17cyclictest14034-21ssh23:04:101
5292995525,12cyclictest18417-21ssh23:09:271
5292995524,18cyclictest15441-21unixbench_multi19:35:351
5292995523,14cyclictest5933-21wget22:15:211
5292995521,17cyclictest29102-21sh22:04:051
5292995518,27cyclictest17893-21sh21:50:251
5292995517,21cyclictest7750irq/56-dwc_otg00:21:021
529199552,38cyclictest0-21swapper/000:24:100
529199551,42cyclictest26584-21grep22:00:300
529199551,38cyclictest9430-21unixbench-2d00:15:350
529199551,27cyclictest9001-21aten_rbpower_en19:20:210
529499542,35cyclictest7850irq/56-dwc_otg_22:06:543
529499542,33cyclictest21478-21ssh23:12:193
529499541,49cyclictest32613-21apt-get20:25:203
529499541,45cyclictest19656-21ssh21:51:533
529499541,42cyclictest27541-21ssh21:22:373
529499541,41cyclictest10030-21ssh23:37:563
529499541,34cyclictest7950irq/56-dwc_otg_00:38:133
529499541,34cyclictest4289-21taskset22:12:563
5292995426,18cyclictest13062-21ssh22:23:521
5292995422,15cyclictest18363-21rm21:11:291
5292995415,12cyclictest0-21swapper/122:50:181
529199549,27cyclictest9-21ksoftirqd/021:55:230
529199543,32cyclictest0-21swapper/019:10:100
5291995424,8cyclictest7750irq/56-dwc_otg00:05:190
529199542,31cyclictest3352-21aten_rbpower_cu23:30:180
529199542,28cyclictest0-21swapper/020:15:310
529499531,41cyclictest18175-21ssh21:11:093
529499531,38cyclictest7950irq/56-dwc_otg_23:52:473
529499531,35cyclictest12405-21taskset00:20:193
5293995316,18cyclictest3847-21/usr/sbin/munin20:35:202
5292995321,13cyclictest31209-21ssh00:03:431
5292995321,12cyclictest22775-21sed19:55:341
5292995320,11cyclictest7950irq/56-dwc_otg_21:50:081
5291995329,14cyclictest0-21swapper/020:25:090
529199532,28cyclictest14734-21aten_rbpower_vo21:05:210
529199531,37cyclictest20951-21ssh21:15:220
529499522,46cyclictest16251-21wget19:40:203
529499521,47cyclictest7168-21/usr/sbin/munin19:15:213
529499521,41cyclictest10268-21runrttasks19:22:133
5292995225,12cyclictest17737-21ssh22:29:511
5292995220,16cyclictest10526-21idleruntime-cro19:25:071
529199522,8cyclictest13576-21aten_rbpower_en21:45:190
5291995226,12cyclictest7750irq/56-dwc_otg23:35:270
529499511,47cyclictest20089-21df_inode19:50:233
529499511,47cyclictest18271-21awk19:45:223
529499511,46cyclictest9208-21apt-get20:50:173
529499511,45cyclictest27091-21apt-get20:10:213
529499511,39cyclictest10730-21taskset19:25:133
5292995124,9cyclictest23483-21apt-get20:00:211
5292995124,17cyclictest6858-21ssh21:36:071
5292995118,26cyclictest3224-1kworker/u9:000:11:121
529199517,25cyclictest8526-21tail20:45:340
529199511,29cyclictest16655-21wget21:10:220
529499501,46cyclictest9732-21ssh00:15:543
5292995019,17cyclictest22928-21dhclient-script00:32:191
5292995019,13cyclictest31517-21systemd-journal19:41:181
529199507,28cyclictest5850-21taskset22:15:200
529199502,27cyclictest0-21swapper/019:15:320
529199501,9cyclictest7750irq/56-dwc_otg19:55:190
5291995010,29cyclictest18364-21sh23:48:030
5292994919,17cyclictest11472-21ssh21:42:011
5292994915,10cyclictest0-21swapper/120:05:261
529199491,35cyclictest0-21swapper/000:25:180
529199491,30cyclictest0-21swapper/023:21:330
529199491,30cyclictest0-21swapper/023:21:330
529499481,44cyclictest29097-21/usr/sbin/munin20:15:233
529499481,43cyclictest5604-21apt-get20:40:203
5293994817,16cyclictest7950irq/56-dwc_otg_21:52:492
5293994812,16cyclictest24001-21apt-get23:55:142
5292994811,22cyclictest7950irq/56-dwc_otg_19:30:171
5291994817,21cyclictest0-21swapper/019:29:300
529199481,24cyclictest31677-21grep23:25:210
5293994715,14cyclictest13562-21/usr/sbin/munin19:30:332
5293994712,15cyclictest9552-21ssh00:15:352
5292994717,9cyclictest0-21swapper/120:16:371
5292994713,10cyclictest7850irq/56-dwc_otg_20:13:361
529199476,25cyclictest0-21swapper/019:45:130
529199471,33cyclictest0-21swapper/020:40:210
529399466,25cyclictest25104-21sh21:20:222
529399463,25cyclictest7950irq/56-dwc_otg_21:13:472
5293994616,15cyclictest301ktimersoftd/222:15:302
5292994623,8cyclictest8636-21chrt19:19:281
5292994618,14cyclictest7021-21taskset20:43:331
529199461,33cyclictest0-21swapper/022:43:200
48322463,13sleep00-21swapper/019:07:400
48322463,13sleep00-21swapper/019:07:400
5293994514,13cyclictest25271-21/usr/sbin/munin20:05:252
5292994520,11cyclictest0-21swapper/120:50:261
529199452,39cyclictest0-21swapper/020:00:390
529399449,18cyclictest7950irq/56-dwc_otg_21:25:282
529399447,25cyclictest0-21swapper/219:45:202
529399447,24cyclictest0-21swapper/219:25:212
529399447,23cyclictest0-21swapper/220:20:212
529399447,23cyclictest0-21swapper/220:15:212
529399445,15cyclictest24564-21ssh21:19:522
529399444,26cyclictest20207-21sh00:29:562
5293994413,19cyclictest32713-21ssh22:46:382
5293994411,11cyclictest30554-21ssh23:23:472
5293994411,11cyclictest30554-21ssh23:23:472
5292994417,11cyclictest0-21swapper/120:25:301
529399439,21cyclictest7891-21ssh23:35:252
529399437,24cyclictest0-21swapper/219:20:212
529399434,23cyclictest7950irq/56-dwc_otg_23:01:232
529399434,15cyclictest7981-21sh22:55:372
529399434,15cyclictest7981-21sh22:55:372
529399431,6cyclictest7950irq/56-dwc_otg_00:35:182
5293994315,13cyclictest23673-21ls22:35:362
5293994312,18cyclictest6624-21ssh00:12:452
5293994311,18cyclictest30804-21ntp_states22:05:322
5292994315,14cyclictest11000-21/usr/sbin/munin20:55:241
5292994315,14cyclictest11000-21/usr/sbin/munin20:55:241
5292994314,14cyclictest0-21swapper/120:49:221
5291994322,11cyclictest7750irq/56-dwc_otg22:45:210
56932420,5sleep25650-21ssh22:15:082
529399427,10cyclictest0-21swapper/222:01:422
529399423,26cyclictest16209-1kworker/u9:021:49:512
529299429,11cyclictest0-21swapper/120:24:491
5292994218,12cyclictest0-21swapper/119:25:131
529399418,25cyclictest0-21swapper/221:57:042
529399416,22cyclictest0-21swapper/219:40:202
5293994111,14cyclictest7239-21sed23:35:082
5292994113,14cyclictest0-21swapper/119:46:301
529199411,18cyclictest0-21swapper/020:10:080
189862410,2sleep2301ktimersoftd/223:49:132
529399408,21cyclictest8873-21seq20:49:072
529399408,12cyclictest12954-21ssh23:40:442
529399407,22cyclictest0-21swapper/220:25:132
529399407,19cyclictest24694-21rm23:15:382
5293994019,5cyclictest1180-21runrttasks23:25:232
5293994010,13cyclictest28810-21sed00:00:322
529399397,22cyclictest12344-21taskset20:57:192
529399397,22cyclictest12344-21taskset20:57:192
529399396,12cyclictest20667-21ssh22:32:272
5293993913,7cyclictest12822-21apt-get21:00:202
5293993913,12cyclictest17541-21ssh22:29:302
5293993912,17cyclictest29779-21ssh22:43:512
5293993910,15cyclictest16890-21ssh23:06:392
529399389,17cyclictest2629-21ssh21:30:552
529399389,17cyclictest2629-21ssh21:30:552
529399389,11cyclictest21709-21ssh23:12:442
529399388,11cyclictest11637-21ssh21:42:192
529399386,18cyclictest2529-21ssh00:08:012
529399385,22cyclictest7950irq/56-dwc_otg_20:10:202
529399374,13cyclictest23168-21sh00:32:462
5293993710,12cyclictest16198-21latency_hist21:10:082
529399368,15cyclictest0-21swapper/220:30:192
5293993615,6cyclictest20584-21awk19:50:312
5293993610,10cyclictest0-21swapper/220:00:212
529399356,14cyclictest16195-21diskmemload00:24:132
529399348,12cyclictest8439-21seq19:17:062
529399345,9cyclictest7950irq/56-dwc_otg_20:00:072
5293993412,15cyclictest7029-21runrttasks20:43:392
529399332,24cyclictest0-21swapper/219:38:232
529399331,14cyclictest0-21swapper/220:55:082
41852324,18sleep10-21swapper/119:05:341
41852324,18sleep10-21swapper/119:05:341
47012273,14sleep20-21swapper/219:05:562
47012273,14sleep20-21swapper/219:05:562
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional