You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 08:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Sat Dec 20, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32733999535,11cyclictest23009-21tail19:48:540
32733999434,10cyclictest10392-21grep23:53:390
32733999236,10cyclictest9383-21unixbench_multi23:48:550
32733999034,11cyclictest18853-21cpuspeed_turbos20:48:460
32741997831,27cyclictest17156-21turbostat19:38:342
32741997730,30cyclictest1419-21runrttasks22:43:462
32741997725,43cyclictest0-21swapper/221:53:482
32741997522,44cyclictest0-21swapper/223:08:562
32741997425,39cyclictest293131munin-node20:04:062
32741997425,39cyclictest293131munin-node20:04:062
32741997424,34cyclictest0-21swapper/221:08:592
32741997419,47cyclictest0-21swapper/220:23:572
32741997324,31cyclictest0-21swapper/220:28:482
32741997226,39cyclictest0-21swapper/222:48:572
32741997121,31cyclictest0-21swapper/219:08:492
32741997120,43cyclictest0-21swapper/222:58:412
32741997120,43cyclictest0-21swapper/222:58:412
32741997027,16cyclictest24773-21threads23:13:562
32741997022,40cyclictest0-21swapper/223:58:392
32741997022,40cyclictest0-21swapper/223:23:552
32741997021,42cyclictest0-21swapper/200:03:362
32741997020,42cyclictest21067-21telnet20:53:442
32741996922,31cyclictest0-21swapper/221:13:592
32741996922,31cyclictest0-21swapper/220:19:072
32741996915,35cyclictest0-21swapper/222:38:532
32741996823,31cyclictest1419-21runrttasks22:32:252
32741996822,40cyclictest0-21swapper/200:32:482
32741996819,33cyclictest0-21swapper/220:14:042
32741996742,18cyclictest31288-21gltestperf23:28:442
32741996734,26cyclictest7041-21grep23:43:562
32741996732,28cyclictest22741-21cut19:48:512
32741996724,36cyclictest18055-21fschecks_count19:38:522
32741996724,35cyclictest25495-21iostat_ios22:08:502
32741996723,29cyclictest0-21swapper/200:28:232
32741996722,29cyclictest13725-21aten_r1power_en22:53:382
32741996722,29cyclictest0-21swapper/221:43:532
32741996718,42cyclictest0-21swapper/219:03:462
32741996715,44cyclictest0-21swapper/219:58:572
32741996630,28cyclictest0-21swapper/221:39:012
32741996626,25cyclictest4130-21munin-node22:33:552
32741996622,37cyclictest0-21swapper/222:03:582
32741996622,25cyclictest26188-21unixbench-2d19:54:062
32741996621,38cyclictest0-21swapper/219:24:362
32741996621,38cyclictest0-21swapper/219:24:362
32741996619,40cyclictest0-21swapper/221:28:422
32741996619,40cyclictest0-21swapper/221:28:422
32741996611,36cyclictest0-21swapper/223:43:212
32741996543,16cyclictest0-21swapper/200:13:452
32741996523,36cyclictest0-21swapper/220:38:522
32741996522,35cyclictest18288-21latency_hist20:48:322
32741996522,35cyclictest18288-21latency_hist20:48:322
32741996522,35cyclictest0-21swapper/219:18:462
32741996520,29cyclictest0-21swapper/220:58:472
32741996518,41cyclictest25655-21aten2.4-expect23:18:372
32741996518,39cyclictest12608-21ntpdc20:33:552
32741996517,41cyclictest0-21swapper/222:24:032
32741996514,44cyclictest0-21swapper/223:53:472
32741996418,29cyclictest0-21swapper/221:38:302
32741996415,42cyclictest0-21swapper/223:03:512
32741996324,32cyclictest1419-21runrttasks19:28:542
32741996321,36cyclictest0-21swapper/222:03:282
32741996320,36cyclictest0-21swapper/219:13:482
32741996319,28cyclictest0-21swapper/200:09:392
32741996318,23cyclictest0-21swapper/221:05:282
32741996312,43cyclictest0-21swapper/221:48:512
32741996224,32cyclictest2737-21latency_hist21:23:302
32741996137,17cyclictest28550-21unixbench_multi22:13:582
32741996137,17cyclictest28550-21unixbench_multi22:13:582
32741996134,12cyclictest1419-21runrttasks20:49:082
32741996125,29cyclictest1604-21latency_hist20:13:332
32741996016,38cyclictest0-21swapper/222:18:442
32741995935,16cyclictest1419-21runrttasks19:44:302
3274399584,21cyclictest0-21swapper/300:14:403
32741995837,16cyclictest0-21swapper/219:03:342
32741995820,12cyclictest2980-21basename21:23:412
32741995816,27cyclictest1287-21snmpd23:35:532
185812583,11sleep10-21swapper/119:39:001
32741995724,26cyclictest0-21swapper/200:18:472
32741995724,26cyclictest0-21swapper/200:18:472
3273899575,15cyclictest5064-21munin-run23:43:241
3273899574,13cyclictest0-21swapper/123:28:581
67472563,8sleep16870-21cat20:23:461
32741995633,16cyclictest8438-21fschecks_time23:48:402
3273899565,32cyclictest0-21swapper/121:34:031
3273899564,35cyclictest0-21swapper/120:10:191
32743995539,9cyclictest16574-21sed00:03:563
3273399553,36cyclictest0-21swapper/019:58:460
248599550,5rtkit-daemon2484-21rtkit-daemon00:33:230
94882543,45sleep39607-21uptime19:19:073
32743995438,10cyclictest19685-21needreboot23:03:513
32743995437,11cyclictest25285-21gltestperf00:23:443
32738995419,12cyclictest0-21swapper/100:02:391
3273399544,44cyclictest0-21swapper/023:08:460
253892543,44sleep125438-21munin-node00:23:471
292342532,9sleep30-21swapper/322:18:313
84092523,42sleep00-21swapper/020:24:220
83832522,41sleep30-21swapper/322:39:513
73262523,42sleep10-21swapper/121:31:091
73262523,42sleep10-21swapper/121:31:091
32743995221,13cyclictest0-21swapper/321:13:493
3273899529,22cyclictest9769-21tr20:28:511
3273899526,28cyclictest0-21swapper/122:28:391
32738995221,13cyclictest0-21swapper/121:08:571
32738995220,9cyclictest17025-21latency_hist00:08:241
3273399523,42cyclictest0-21swapper/021:08:450
302462524,12sleep30-21swapper/321:13:303
170692522,9sleep30-21swapper/319:37:373
149882525,9sleep00-21swapper/019:33:430
121712523,42sleep10-21swapper/119:26:031
121712523,42sleep10-21swapper/119:26:031
10842522,9sleep30-21swapper/322:25:003
3274399514,40cyclictest0-21swapper/321:43:243
3274399514,29cyclictest0-21swapper/321:43:573
3274399514,29cyclictest0-21swapper/320:43:413
3274399514,29cyclictest0-21swapper/320:43:413
32743995135,10cyclictest9830-21gltestperf20:28:523
3273899515,27cyclictest0-21swapper/122:18:481
3273899515,27cyclictest0-21swapper/120:48:561
3273899515,27cyclictest0-21swapper/120:33:571
3273899514,27cyclictest0-21swapper/122:03:401
3273399514,41cyclictest0-21swapper/021:03:320
3273399514,41cyclictest0-21swapper/020:43:570
3273399514,41cyclictest0-21swapper/020:43:570
313732512,42sleep30-21swapper/320:07:283
313732512,42sleep30-21swapper/320:07:283
300192513,42sleep10-21swapper/123:26:321
288072513,42sleep10-21swapper/122:15:511
288072513,42sleep10-21swapper/122:15:511
28132513,9sleep00-21swapper/019:08:280
264352512,9sleep30-21swapper/319:55:153
167472515,9sleep00-21swapper/021:49:260
155112512,9sleep30-21swapper/322:54:183
86822503,9sleep30-21swapper/320:27:563
3274399509,25cyclictest0-21swapper/300:08:343
3274399504,28cyclictest0-21swapper/321:58:593
3274399504,27cyclictest0-21swapper/321:53:293
3274399503,29cyclictest0-21swapper/320:38:473
32743995022,20cyclictest3588-21sort23:38:433
3273899504,27cyclictest0-21swapper/119:48:531
3273899504,27cyclictest0-21swapper/119:18:511
3273899504,25cyclictest12301-21munin-run21:43:291
32738995036,8cyclictest5764-21cut19:13:461
32738995034,8cyclictest3858-21tail21:23:491
32738995031,12cyclictest3400-21cpuspeed_turbos19:08:501
3273399506,39cyclictest0-21swapper/022:34:160
3273399504,38cyclictest0-21swapper/022:23:430
3273399503,41cyclictest0-21swapper/023:07:320
3273399503,41cyclictest0-21swapper/021:41:240
32733995023,12cyclictest0-21swapper/022:13:470
32733995023,12cyclictest0-21swapper/022:13:470
32733995023,12cyclictest0-21swapper/000:08:460
265962505,9sleep00-21swapper/019:57:210
25512502,9sleep30-21swapper/323:37:063
249882502,41sleep00-21swapper/000:23:420
191082503,9sleep10-21swapper/100:09:361
180202503,39sleep10-21swapper/121:53:461
173712502,41sleep30-21swapper/319:38:383
170822503,41sleep10-21swapper/119:37:481
157252502,41sleep00-21swapper/022:56:490
156952502,41sleep00-21swapper/020:40:590
10012503,10sleep10-21swapper/122:24:051
98572494,9sleep00-21swapper/021:38:020
71812492,40sleep00-21swapper/023:45:050
49762492,40sleep00-21swapper/021:26:370
37022495,9sleep00-21swapper/022:31:010
3274399494,37cyclictest0-21swapper/323:56:433
3274399494,27cyclictest0-21swapper/322:33:553
3274399494,27cyclictest0-21swapper/322:28:513
3274399494,27cyclictest0-21swapper/322:03:533
3274399494,27cyclictest0-21swapper/320:09:013
3274399494,26cyclictest0-21swapper/322:48:273
32743994932,10cyclictest26487-21tail23:18:453
32743994932,10cyclictest19330-21gltestperf20:48:523
32743994931,10cyclictest5122-21crond21:28:293
3274399493,27cyclictest0-21swapper/323:48:243
3274399493,27cyclictest0-21swapper/321:33:393
32743994921,11cyclictest10245-21telnet19:23:453
32743994921,11cyclictest10245-21telnet19:23:453
32743994920,12cyclictest0-21swapper/320:13:443
32743994920,12cyclictest0-21swapper/319:28:553
32743994918,13cyclictest0-21swapper/319:08:563
3273899495,26cyclictest0-21swapper/121:48:591
3273899495,26cyclictest0-21swapper/100:30:331
3273899495,25cyclictest0-21swapper/119:43:491
3273899494,28cyclictest0-21swapper/120:58:541
3273899494,27cyclictest0-21swapper/123:48:541
3273899494,27cyclictest0-21swapper/123:33:591
3273899494,27cyclictest0-21swapper/121:59:031
3273899494,26cyclictest0-21swapper/120:04:051
3273899494,26cyclictest0-21swapper/120:04:051
3273899493,40cyclictest1419-21runrttasks21:18:351
32738994919,12cyclictest0-21swapper/119:53:531
3273399494,28cyclictest0-21swapper/020:04:000
3273399494,28cyclictest0-21swapper/020:04:000
3273399493,41cyclictest0-21swapper/019:10:450
3273399493,39cyclictest0-21swapper/021:53:550
277982492,41sleep00-21swapper/021:07:510
241912492,9sleep30-21swapper/319:52:213
238492492,41sleep00-21swapper/000:19:100
238492492,41sleep00-21swapper/000:19:100
232982493,9sleep30-21swapper/320:58:353
18382492,40sleep00-21swapper/023:33:570
173782493,9sleep00-21swapper/019:38:430
120822493,40sleep10-21swapper/123:56:221
37092482,40sleep00-21swapper/020:15:190
3274399484,27cyclictest0-21swapper/322:48:553
3274399484,27cyclictest0-21swapper/319:59:033
3274399484,26cyclictest0-21swapper/323:48:493
3274399484,26cyclictest0-21swapper/323:13:583
3274399484,26cyclictest0-21swapper/321:54:033
3274399484,26cyclictest0-21swapper/319:08:353
32743994831,10cyclictest6064-21fschecks_count19:13:503
3273899484,38cyclictest0-21swapper/123:08:251
3273899484,26cyclictest0-21swapper/123:11:371
32738994833,9cyclictest28427-21grep19:59:051
32738994830,10cyclictest20490-21grep00:13:441
3273899483,29cyclictest0-21swapper/122:09:011
3273899483,28cyclictest0-21swapper/121:43:321
32738994820,8cyclictest5290-21munin-node23:43:341
32738994819,10cyclictest30473-21munin-node21:13:501
32738994818,12cyclictest0-21swapper/120:14:001
3273399484,38cyclictest0-21swapper/022:48:260
3273399484,38cyclictest0-21swapper/022:09:010
3273399483,39cyclictest0-21swapper/021:28:500
3273399483,39cyclictest0-21swapper/021:28:500
3273399483,20cyclictest0-21swapper/019:13:440
287712482,9sleep30-21swapper/322:15:253
287712482,9sleep30-21swapper/322:15:253
26452483,38sleep10-21swapper/119:06:161
179082483,9sleep10-21swapper/122:59:111
179082483,9sleep10-21swapper/122:59:111
156992484,9sleep10-21swapper/122:56:291
3274399474,26cyclictest0-21swapper/323:08:543
3274399474,26cyclictest0-21swapper/320:53:543
32743994731,10cyclictest21491-21unixbench_singl19:44:063
32743994718,12cyclictest0-21swapper/323:28:523
32743994716,13cyclictest0-21swapper/323:23:473
3273899474,26cyclictest0-21swapper/122:48:411
3273899474,26cyclictest0-21swapper/120:40:241
3273899474,13cyclictest0-21swapper/123:18:591
3273899474,11cyclictest24936-21unixbench_multi23:13:581
32738994730,11cyclictest17000-21gltestperf20:43:491
32738994730,11cyclictest17000-21gltestperf20:43:491
3273899473,39cyclictest0-21swapper/122:44:261
3273399479,33cyclictest41ktimersoftd/023:13:560
3273399475,36cyclictest0-21swapper/021:59:000
32733994732,9cyclictest4503-21grep20:18:460
32733994732,9cyclictest22455-21grep22:03:440
248599470,4rtkit-daemon2484-21rtkit-daemon00:05:020
24582473,39sleep00-21swapper/021:20:240
218092473,38sleep00-21swapper/019:47:220
12262472,38sleep00-21swapper/020:11:000
61732463,37sleep10-21swapper/122:37:071
32743994632,9cyclictest13158-21fschecks_count23:58:423
3274399463,27cyclictest0-21swapper/321:03:463
32743994617,8cyclictest121850irq/46-eth021:28:493
32743994617,8cyclictest121850irq/46-eth021:28:493
3273899464,25cyclictest0-21swapper/121:05:281
3273899464,22cyclictest21418-21sshd20:53:471
32738994631,10cyclictest22028-21munin-node00:18:441
32738994631,10cyclictest22028-21munin-node00:18:441
32738994628,11cyclictest5043-21gltestperf20:18:511
32738994616,13cyclictest0-21swapper/119:28:481
3273399463,36cyclictest0-21swapper/023:28:400
3273399463,11cyclictest0-21swapper/019:29:030
32733994620,11cyclictest0-21swapper/023:39:410
32733994619,13cyclictest0-21swapper/021:14:550
32733994618,22cyclictest27780-21latency_hist23:23:250
32733994617,12cyclictest0-21swapper/020:33:310
143432463,9sleep00-21swapper/021:44:140
41842452,9sleep30-21swapper/320:18:373
3274399455,13cyclictest0-21swapper/300:28:383
3274399454,35cyclictest0-21swapper/322:58:423
3274399454,35cyclictest0-21swapper/322:58:423
3273899454,35cyclictest7979-21switchtime22:38:571
3273399458,31cyclictest0-21swapper/020:53:470
3273399454,35cyclictest0-21swapper/000:03:230
32733994527,12cyclictest0-21swapper/022:38:500
32733994524,9cyclictest16115-21telnet22:58:360
32733994524,9cyclictest16115-21telnet22:58:360
32733994511,26cyclictest0-21swapper/019:18:470
3274399444,24cyclictest0-21swapper/322:08:403
3274399444,22cyclictest0-21swapper/300:18:343
3274399444,22cyclictest0-21swapper/300:18:343
32743994428,10cyclictest1194-21fschecks_time21:18:483
32733994432,8cyclictest28862-21gltestperf23:23:450
3273399443,35cyclictest0-21swapper/022:18:410
3274399434,24cyclictest0-21swapper/320:33:423
3273399433,34cyclictest12500-21strings22:48:520
3273399433,12cyclictest0-21swapper/019:23:570
3273399433,12cyclictest0-21swapper/019:23:570
305542434,32sleep10-21swapper/118:58:441
3273399424,33cyclictest0-21swapper/020:33:430
3273399423,26cyclictest0-21swapper/000:13:360
3273399353,26cyclictest0-21swapper/019:03:340
32743993220,7cyclictest361ktimersoftd/319:03:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional