You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 08:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Apr 06, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
220962960,2sleep20-21swapper/209:50:572
5332999525,18cyclictest12011-21ssh08:20:480
12572940,3sleep2534499cyclictest05:46:072
120262920,2sleep20-21swapper/208:20:502
17802840,2sleep31779-21ssh06:50:573
276632820,5sleep0533299cyclictest08:39:320
5332997919,48cyclictest0-21swapper/006:00:490
3282790,5sleep3534799cyclictest05:45:563
154492790,2sleep20-21swapper/209:03:382
533999785,12cyclictest0-21swapper/109:45:511
5332997824,10cyclictest0-21swapper/007:40:500
5332997824,10cyclictest0-21swapper/007:40:500
533299779,20cyclictest7869-21sh08:15:490
273872770,5sleep2534499cyclictest09:56:282
5339997537,12cyclictest1819-21/usr/sbin/munin09:26:021
5339997336,13cyclictest12742-21aten_rbpower_cu04:50:511
5339997333,16cyclictest17330-21if_err_eth009:05:571
201882730,3sleep020187-21tune2fs05:10:550
173182730,5sleep0533299cyclictest09:45:390
173182730,5sleep0533299cyclictest09:45:390
533299726,45cyclictest9-21ksoftirqd/007:50:510
5339997131,14cyclictest1-21systemd07:21:051
5339997131,14cyclictest1-21systemd07:21:051
162322710,2sleep10-21swapper/107:07:051
94532700,3sleep111-21rcu_preempt07:38:051
534499708,12cyclictest7850irq/56-dwc_otg_08:55:512
5339997033,15cyclictest21051-21ssh07:13:071
5339996931,24cyclictest17097-21sh09:45:201
5339996931,24cyclictest17097-21sh09:45:201
5339996930,19cyclictest31310-21rm09:22:201
5339996833,19cyclictest27521-21ssh06:42:551
5339996731,19cyclictest26307-21sed05:26:041
533299676,7cyclictest0-21swapper/006:50:510
5339996633,20cyclictest10060-21ssh08:17:421
5339996631,20cyclictest26743-21ssh09:16:341
5339996622,27cyclictest7950irq/56-dwc_otg_09:31:261
533299668,54cyclictest12540-21apt-get06:20:390
533299668,54cyclictest12540-21apt-get06:20:390
533299663,53cyclictest15979-21kworker/0:106:44:580
327482660,3sleep132738-21ssh08:45:491
242212660,5sleep3534799cyclictest05:21:003
5339996534,22cyclictest2019-21sort06:50:591
5339996438,19cyclictest15156-21ssh09:03:061
5339996432,15cyclictest17552-21ssh08:26:141
5339996429,21cyclictest27436-21ssh09:56:331
5339996428,14cyclictest27692-21ssh08:00:421
5339996427,24cyclictest24658-21sh09:54:021
5339996338,12cyclictest22198-21ssh08:32:101
5339996335,18cyclictest17011-21ssh07:46:461
5339996325,10cyclictest7950irq/56-dwc_otg_08:40:481
5339996324,20cyclictest13961-21ssh07:05:351
273442630,2sleep20-21swapper/208:38:582
5344996221,16cyclictest7950irq/56-dwc_otg_06:40:522
5344996220,22cyclictest7850irq/56-dwc_otg_05:30:522
5344996220,22cyclictest7850irq/56-dwc_otg_05:30:522
5339996236,8cyclictest3751-21apt-get05:55:521
5339996233,12cyclictest6994-21ssh08:14:511
5339996232,15cyclictest24067-21latency05:20:581
5339996231,17cyclictest22993-21ssh07:55:061
5339996231,11cyclictest15366-21/usr/sbin/munin06:26:001
5339996230,19cyclictest29469-21rm06:45:451
5339996227,15cyclictest1998-21aten_rbpower_en05:50:521
5339996226,13cyclictest23345-21latency_hist05:20:391
5339996223,24cyclictest24417-21ssh09:14:531
533299622,29cyclictest0-21swapper/008:50:500
270282620,2sleep00-21swapper/007:59:550
5339996133,17cyclictest5679-21ssh06:55:491
5339996130,14cyclictest13831-21ssh07:43:311
5339996130,14cyclictest13831-21ssh07:43:311
5339996128,22cyclictest745-21ssh07:27:261
5339996128,11cyclictest0-21swapper/107:35:391
5339996127,11cyclictest7850irq/56-dwc_otg_04:45:561
5344996022,20cyclictest0-21swapper/208:00:522
5339996034,8cyclictest23002-21df_inode07:15:521
5339996029,12cyclictest11632-21ntpq06:15:591
5339996029,12cyclictest11632-21ntpq06:15:591
5339996027,16cyclictest26163-21ssh07:58:191
5339996022,13cyclictest0-21swapper/109:00:381
533299601,34cyclictest30748-21munin-run07:25:390
533299601,34cyclictest30748-21munin-run07:25:390
5339995931,14cyclictest12817-21ssh09:40:041
5339995927,14cyclictest6333-21ssh08:52:071
533299592,37cyclictest22330-21unixbench_singl09:11:070
5332995913,11cyclictest0-21swapper/006:10:510
5339995833,16cyclictest23788-21ssh06:38:531
5339995824,10cyclictest7750irq/56-dwc_otg05:15:391
5339995821,24cyclictest0-21swapper/106:34:561
5332995831,5cyclictest7850irq/56-dwc_otg_08:06:050
5339995732,13cyclictest28611-21chrt05:34:301
5339995732,13cyclictest28611-21chrt05:34:301
5339995727,11cyclictest12465-21if_eth008:20:571
533299577,25cyclictest0-21swapper/004:40:520
533299571,27cyclictest27969-21ls08:00:530
533299571,27cyclictest11365-21unixbench_singl07:01:080
5339995627,10cyclictest0-21swapper/106:15:171
5339995625,16cyclictest28404-21date08:40:391
5339995625,12cyclictest946-21ssh08:06:241
5339995623,10cyclictest0-21swapper/105:50:381
5339995621,9cyclictest7750irq/56-dwc_otg06:07:111
5347995533,9cyclictest7950irq/56-dwc_otg_08:01:073
5339995531,11cyclictest12874-21awk06:20:511
5339995529,11cyclictest19396-21runrttasks05:07:041
5339995523,11cyclictest7950irq/56-dwc_otg_04:45:381
533299558,18cyclictest8986-21cut08:55:560
5332995528,7cyclictest13875-21unixbench_singl04:51:060
5332995524,7cyclictest0-21swapper/005:31:080
5332995524,7cyclictest0-21swapper/005:31:080
5332995524,14cyclictest5845-21ssh09:31:020
5344995421,10cyclictest1011-21apt-get09:25:492
5339995427,13cyclictest30425-21chrt05:39:341
5332995429,5cyclictest32346-21unixbench-2d07:26:070
533299542,38cyclictest0-21swapper/006:56:060
533299542,30cyclictest0-21swapper/006:35:540
533299542,28cyclictest0-21swapper/004:35:520
5332995419,27cyclictest7950irq/56-dwc_otg_07:20:390
5339995321,9cyclictest0-21swapper/104:33:301
5332995328,5cyclictest29418-21aten_rbpower_po09:20:520
534499522,7cyclictest7950irq/56-dwc_otg_07:00:492
5339995222,12cyclictest7750irq/56-dwc_otg05:04:281
5332995226,5cyclictest0-21swapper/009:55:380
5332995225,6cyclictest7850irq/56-dwc_otg_09:36:030
533299521,31cyclictest4722-21grep08:11:050
533299521,28cyclictest30769-21grep06:46:070
533299521,24cyclictest2923-21timerwakeupswit05:51:060
534799515,5cyclictest0-21swapper/304:40:523
5344995113,18cyclictest12883-21apt-get06:20:512
5339995126,11cyclictest8727-21chrt04:40:281
5339995122,12cyclictest32055-21chrt05:42:151
533299512,28cyclictest1698-21ls05:50:380
5332995117,18cyclictest7850irq/56-dwc_otg_07:11:070
533299511,47cyclictest0-21swapper/005:40:510
5332995112,20cyclictest28888-21fschecks_count08:40:540
5344995010,21cyclictest0-21swapper/205:35:532
5339995021,13cyclictest1-21systemd04:58:581
5332995028,2cyclictest0-21swapper/005:00:500
5332995026,5cyclictest4503-21sshd05:56:050
5332995022,8cyclictest0-21swapper/008:45:500
533299501,8cyclictest25577-21systemd-journal09:46:080
533299501,46cyclictest20581-21grep08:30:550
5347994929,5cyclictest16915-21sh08:26:013
5339994932,12cyclictest7165-21latency_hist06:05:391
533299499,2cyclictest0-21swapper/009:25:490
533299495,26cyclictest0-21swapper/005:05:560
5332994936,3cyclictest0-21swapper/006:25:390
533299492,27cyclictest0-21swapper/004:55:560
5344994814,9cyclictest0-21swapper/205:15:502
5332994822,7cyclictest12731-21/usr/sbin/munin09:01:040
533299481,25cyclictest15415-21aten_rbpower_po07:45:510
534799479,20cyclictest0-21swapper/308:31:073
534499479,12cyclictest15413-21/usr/sbin/munin06:26:022
534499477,25cyclictest0-21swapper/205:20:512
5344994712,16cyclictest11625-21ssh08:20:332
533299479,5cyclictest0-21swapper/009:56:070
5332994726,3cyclictest0-21swapper/007:06:050
5332994718,21cyclictest0-21swapper/005:26:350
5332994711,4cyclictest20730-21idleruntime-cro09:10:390
534799467,25cyclictest0-21swapper/306:20:443
533299469,26cyclictest0-21swapper/007:35:520
5332994626,5cyclictest7750irq/56-dwc_otg04:45:440
533299462,24cyclictest22172-21awk05:15:560
5332994620,6cyclictest25577-21systemd-journal06:30:380
533299461,34cyclictest28096-21ssh09:19:030
534799457,23cyclictest0-21swapper/307:40:523
534799457,23cyclictest0-21swapper/307:40:523
534799452,11cyclictest32750-21ssh08:45:503
534499457,8cyclictest7850irq/56-dwc_otg_07:05:522
534499456,25cyclictest28618-21ssh08:40:512
534499454,5cyclictest0-21swapper/207:15:482
5344994522,15cyclictest7750irq/56-dwc_otg07:29:592
534499452,10cyclictest7950irq/56-dwc_otg_07:55:502
5344994514,20cyclictest6620-21ssh07:35:322
533299451,24cyclictest0-21swapper/005:35:590
533299451,14cyclictest0-21swapper/006:31:060
534799447,23cyclictest0-21swapper/304:35:523
534799447,22cyclictest0-21swapper/306:10:523
534499449,12cyclictest25998-21/usr/sbin/munin05:26:002
534499445,11cyclictest7950irq/56-dwc_otg_05:56:052
534499443,25cyclictest7950irq/56-dwc_otg_06:59:182
5344994416,13cyclictest19662-21ssh09:08:452
5344994413,8cyclictest13081-21if_err_eth004:50:562
534499441,24cyclictest7950irq/56-dwc_otg_07:38:282
5344994412,17cyclictest11279-21ssh09:37:172
533299443,21cyclictest3551-21ssh07:31:000
5332994420,5cyclictest16300-21wget08:25:530
48712443,12sleep00-21swapper/004:27:590
534799433,7cyclictest21111-21wget09:10:523
5344994313,19cyclictest31916-21rm09:23:272
5344994312,23cyclictest17247-21ssh07:47:102
533299432,26cyclictest0-21swapper/004:33:100
5332994319,3cyclictest0-21swapper/006:05:540
5332994317,14cyclictest0-21swapper/005:20:510
534799429,16cyclictest4466-21ssh08:50:503
534799427,22cyclictest0-21swapper/304:50:513
534499427,25cyclictest0-21swapper/206:50:522
534499427,13cyclictest12762-21ssh07:41:352
534499427,13cyclictest12762-21ssh07:41:352
534499426,24cyclictest0-21swapper/206:30:522
5344994213,12cyclictest7015-21sh08:14:542
5344994213,11cyclictest27725-21ssh07:21:052
5344994213,11cyclictest27725-21ssh07:21:052
5344994210,22cyclictest951-21ssh08:06:252
534799417,19cyclictest0-21swapper/307:35:523
534499419,13cyclictest32054-21rm06:48:122
534499416,12cyclictest28893-21ssh09:20:302
534499415,11cyclictest0-21swapper/207:51:002
5344994117,5cyclictest301ktimersoftd/208:46:022
5344994114,6cyclictest13536-21aten_rbpower_en09:40:522
5344994114,6cyclictest13536-21aten_rbpower_en09:40:522
5344994114,17cyclictest23385-21ssh08:34:202
5344994113,16cyclictest4596-21wget08:50:532
5344994113,11cyclictest17588-21sshd05:01:592
5344994110,11cyclictest24404-21rm09:14:522
534799407,18cyclictest7950irq/56-dwc_otg_09:00:393
5347994020,5cyclictest3518-21idleruntime-cro05:55:383
534499406,12cyclictest19307-21ssh08:29:272
534499405,24cyclictest7555-21ssh09:33:032
534499405,12cyclictest1976-21aten_rbpower_en05:50:512
5347993924,10cyclictest381ktimersoftd/307:20:523
5347993924,10cyclictest381ktimersoftd/307:20:523
5347993919,17cyclictest0-21swapper/305:30:523
5347993919,17cyclictest0-21swapper/305:30:523
5347993912,4cyclictest0-21swapper/308:15:393
534799386,21cyclictest0-21swapper/307:25:523
534499389,11cyclictest9181-21aten_rbpower_cu06:10:522
534499388,16cyclictest0-21swapper/207:10:442
534499387,17cyclictest11138-21fschecks_count04:45:562
534499386,17cyclictest24584-21ssh06:40:162
534499385,23cyclictest7950irq/56-dwc_otg_09:45:512
534499381,25cyclictest7938-21ls06:05:582
5339993822,12cyclictest5341-21cat04:30:391
41442380,5sleep3534799cyclictest07:31:073
534799375,4cyclictest7850irq/56-dwc_otg_09:21:013
5347993728,6cyclictest26009-21ssh08:36:343
534799371,4cyclictest7850irq/56-dwc_otg_09:15:563
5347993713,4cyclictest25373-21aten_rbpower_cu05:25:523
534499374,20cyclictest32061-21taskset05:42:202
534499371,20cyclictest7950irq/56-dwc_otg_05:15:392
534799361,20cyclictest19808-21fschecks_count07:50:553
5347993610,4cyclictest0-21swapper/307:45:523
534499368,10cyclictest0-21swapper/205:07:292
5344993616,5cyclictest7144-21idleruntime-cro06:05:392
534799358,5cyclictest7850irq/56-dwc_otg_04:45:523
534799358,14cyclictest7950irq/56-dwc_otg_09:26:063
534799357,23cyclictest0-21swapper/305:40:513
534799353,3cyclictest0-21swapper/305:05:503
534799346,5cyclictest21998-21cut05:15:543
534799346,21cyclictest0-21swapper/307:00:523
534799343,6cyclictest0-21swapper/304:56:063
5347993414,4cyclictest21815-21/usr/sbin/munin06:36:063
5347993412,6cyclictest22122-21sort09:50:573
534799341,21cyclictest29698-21latency_hist10:00:393
534499345,5cyclictest6489-21unixbench_multi04:31:052
31822340,5sleep3534799cyclictest08:10:293
534799336,16cyclictest7950irq/56-dwc_otg_08:24:203
5347993324,7cyclictest39-21ksoftirqd/306:46:083
5347993312,4cyclictest0-21swapper/309:40:493
5347993312,4cyclictest0-21swapper/309:40:493
534799331,17cyclictest15384-21kworker/3:209:06:073
534499334,15cyclictest0-21swapper/204:40:322
5344993312,12cyclictest10979-21basename06:15:512
5344993312,12cyclictest10979-21basename06:15:512
534799329,11cyclictest0-21swapper/309:36:013
534799328,4cyclictest7850irq/56-dwc_otg_07:56:033
534799327,5cyclictest7750irq/56-dwc_otg09:31:083
534799325,7cyclictest9873-21apt-config07:00:403
534799321,11cyclictest30115-21unixbench_singl05:36:083
5347993211,10cyclictest16241-21diskmemload09:46:043
534799317,15cyclictest0-21swapper/307:10:523
534799315,5cyclictest18460-21ssh07:10:393
534799313,20cyclictest0-21swapper/304:32:483
534799312,3cyclictest7950irq/56-dwc_otg_07:15:483
534799311,11cyclictest32564-21ssh08:45:403
534499313,15cyclictest7950irq/56-dwc_otg_04:42:552
534499311,11cyclictest15943-21seq04:59:032
534799309,5cyclictest7850irq/56-dwc_otg_06:06:013
534799307,11cyclictest0-21swapper/308:20:153
534799304,6cyclictest12539-21latency_hist06:20:393
534799304,6cyclictest12539-21latency_hist06:20:393
534799302,9cyclictest0-21swapper/306:34:053
534799294,4cyclictest7950irq/56-dwc_otg_06:25:523
534799292,17cyclictest0-21swapper/306:00:493
5347992912,11cyclictest0-21swapper/309:00:493
48382294,16sleep30-21swapper/304:27:343
534799282,4cyclictest7750irq/56-dwc_otg06:40:533
534799281,4cyclictest0-21swapper/305:00:563
534799281,21cyclictest21498-21latency_hist05:15:393
48322283,15sleep20-21swapper/204:27:292
534799271,20cyclictest0-21swapper/305:58:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional