You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-27 - 09:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Wed Aug 27, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2868899944,26cyclictest0-21swapper/022:14:180
2868899944,26cyclictest0-21swapper/022:14:180
223099930,6rtkit-daemon0-21swapper/119:31:251
223099930,6rtkit-daemon0-21swapper/119:14:331
223099920,7rtkit-daemon0-21swapper/122:02:041
2868899914,26cyclictest0-21swapper/019:13:160
2868899904,64cyclictest0-21swapper/022:24:130
2868899904,27cyclictest0-21swapper/021:09:160
28688998932,50cyclictest7780-21grep22:34:170
28688998927,56cyclictest0-21swapper/017:49:180
223099890,12rtkit-daemon2229-21rtkit-daemon18:13:282
223099860,5rtkit-daemon2229-21rtkit-daemon21:04:212
2868899853,25cyclictest22510-21latency_hist23:08:410
2868899843,73cyclictest0-21swapper/020:49:130
2868899843,73cyclictest0-21swapper/020:49:130
28688998418,12cyclictest0-21swapper/023:08:570
2868899835,25cyclictest0-21swapper/020:54:010
2868899834,71cyclictest0-21swapper/017:54:230
2868899828,10cyclictest17109-21taskset20:39:190
2868899824,26cyclictest0-21swapper/018:49:150
2868899823,72cyclictest0-21swapper/018:09:130
2868899823,23cyclictest3652-21telnet22:28:550
2868899818,8cyclictest20248-21crond18:33:510
2868899814,70cyclictest0-21swapper/022:08:120
2868899814,70cyclictest0-21swapper/019:34:130
2868899813,26cyclictest0-21swapper/018:24:160
2868899813,11cyclictest4617-21grep17:59:130
2868899813,11cyclictest4617-21grep17:59:130
28688998126,48cyclictest0-21swapper/019:54:100
28688998024,12cyclictest10970-21tail18:14:040
28688998023,8cyclictest1265-21runrttasks18:07:170
28688998022,8cyclictest1265-21runrttasks21:43:280
28688998021,8cyclictest17967-21missed_timers21:49:120
28688998020,10cyclictest14006-21grep22:49:010
28688998013,14cyclictest0-21swapper/021:04:090
2868899795,30cyclictest0-21swapper/022:23:430
2868899794,24cyclictest0-21swapper/021:23:580
2868899793,68cyclictest0-21swapper/020:19:120
28688997926,11cyclictest3093-21grep20:09:250
28688997921,8cyclictest1265-21runrttasks18:41:260
2868899784,23cyclictest20979-21grep19:39:240
2868899784,11cyclictest0-21swapper/021:14:220
28688997828,43cyclictest20510-21aten2.4-expect18:34:030
28688997823,8cyclictest16300-21grep22:54:000
28688997823,8cyclictest16300-21grep22:54:000
28688997819,10cyclictest1160-21snmpd20:30:160
2868899774,27cyclictest0-21swapper/020:44:040
2868899774,24cyclictest0-21swapper/020:24:040
2868899774,23cyclictest0-21swapper/018:44:110
2868899774,23cyclictest0-21swapper/018:44:110
2868899774,16cyclictest0-21swapper/022:58:550
2868899774,16cyclictest0-21swapper/022:09:020
2868899773,29cyclictest0-21swapper/022:43:570
2868899773,26cyclictest0-21swapper/018:19:270
2868899773,22cyclictest27350-21crond21:03:460
28688997715,9cyclictest0-21swapper/021:45:270
2868899764,14cyclictest0-21swapper/019:49:040
2868899763,12cyclictest7315-21munin-node19:14:220
28688997621,7cyclictest1265-21runrttasks19:22:340
28688997613,9cyclictest1160-21snmpd20:17:140
28688997612,16cyclictest0-21swapper/021:34:190
28688997612,16cyclictest0-21swapper/021:34:190
2868899754,12cyclictest0-21swapper/020:34:160
2868899753,24cyclictest0-21swapper/019:44:010
2868899753,20cyclictest0-21swapper/020:04:000
2868899753,20cyclictest0-21swapper/020:04:000
2868899753,17cyclictest0-21swapper/019:29:020
28688997522,7cyclictest1265-21runrttasks21:57:310
28688997513,8cyclictest32441-21munin-node18:59:050
2868899744,30cyclictest0-21swapper/018:54:190
2868899743,26cyclictest1160-21snmpd19:25:060
2868899743,26cyclictest1160-21snmpd19:25:060
2868899743,20cyclictest1160-21snmpd19:06:160
28688997422,7cyclictest8442-21aten_r1power_cu22:38:540
2868899734,28cyclictest0-21swapper/019:59:210
2868899734,23cyclictest0-21swapper/017:44:100
2868899733,25cyclictest0-21swapper/021:19:070
28688997219,8cyclictest7552-21grep21:28:580
28688997219,8cyclictest7552-21grep21:28:580
2868899713,27cyclictest0-21swapper/022:00:300
300532680,4sleep130122-21processes22:14:171
300532680,4sleep130122-21processes22:14:171
223099670,4rtkit-daemon2229-21rtkit-daemon22:28:562
2868999634,29cyclictest0-21swapper/121:54:221
2868999634,12cyclictest2588-21head21:18:581
264802634,13sleep20-21swapper/222:08:572
28689996219,36cyclictest0-21swapper/119:34:041
212862624,12sleep20-21swapper/219:42:102
223099610,12rtkit-daemon2229-21rtkit-daemon18:15:133
28690996041,10cyclictest28149-21grep19:54:222
2868999604,49cyclictest0-21swapper/121:28:451
2868999604,29cyclictest0-21swapper/118:28:511
28689996024,14cyclictest0-21swapper/117:44:211
28691995940,12cyclictest0-21swapper/317:54:073
2869199594,47cyclictest0-21swapper/322:23:423
28690995838,12cyclictest7377-21needreboot18:04:192
2868999585,26cyclictest1265-21runrttasks20:14:211
2868999583,48cyclictest0-21swapper/120:39:111
172412573,47sleep317381-21df_inode21:49:033
2869099564,32cyclictest0-21swapper/221:54:122
28690995638,11cyclictest30300-21cpuspeed_turbos21:09:012
28690995638,11cyclictest21885-21tr21:59:012
2868999565,31cyclictest0-21swapper/119:04:051
28689995636,12cyclictest6137-21needreboot19:09:211
2869199554,44cyclictest0-21swapper/318:34:043
28691995535,13cyclictest0-21swapper/322:40:363
2868999553,45cyclictest11493-21kworker/u8:120:24:131
2868999553,13cyclictest0-21swapper/122:04:151
2869199544,44cyclictest0-21swapper/319:14:023
28691995438,10cyclictest5307-21tr19:09:103
2869099544,30cyclictest0-21swapper/219:29:012
28690995433,14cyclictest0-21swapper/218:40:212
223099540,6rtkit-daemon2229-21rtkit-daemon17:39:222
223099540,5rtkit-daemon2229-21rtkit-daemon22:04:043
223099540,5rtkit-daemon2229-21rtkit-daemon21:40:353
199932543,11sleep10-21swapper/120:46:131
161652543,44sleep216267-21grep22:54:002
161652543,44sleep216267-21grep22:54:002
308682533,44sleep10-21swapper/120:02:241
28691995341,7cyclictest18563-21unixbench_multi19:34:243
2869199534,30cyclictest913-21dbus-daemon20:53:463
2869199534,30cyclictest913-21dbus-daemon20:53:463
28691995337,10cyclictest15773-21head18:24:043
28691995315,23cyclictest18099-21munin-node18:29:273
2869099534,31cyclictest0-21swapper/218:19:082
28690995336,11cyclictest5817-21grep19:09:162
28690995324,21cyclictest0-21swapper/219:19:152
2868999535,28cyclictest0-21swapper/118:54:141
2868999533,36cyclictest0-21swapper/120:38:461
260412532,9sleep30-21swapper/319:51:463
223099530,5rtkit-daemon2229-21rtkit-daemon23:03:543
223099530,5rtkit-daemon2229-21rtkit-daemon17:59:063
223099530,5rtkit-daemon2229-21rtkit-daemon17:59:063
211762533,10sleep10-21swapper/119:40:441
143522533,11sleep10-21swapper/118:19:181
2869199524,29cyclictest0-21swapper/322:53:413
2869199524,28cyclictest0-21swapper/320:13:473
2869199523,39cyclictest0-21swapper/322:59:013
2869199522,43cyclictest0-21swapper/319:24:023
2869199522,43cyclictest0-21swapper/319:24:023
28690995234,10cyclictest30980-21grep18:54:132
2868999524,38cyclictest0-21swapper/119:54:011
28689995220,13cyclictest0-21swapper/118:14:161
259282523,12sleep10-21swapper/119:50:191
223099520,5rtkit-daemon2229-21rtkit-daemon21:13:213
54572512,43sleep30-21swapper/322:29:343
321582512,44sleep132214-21latency22:19:121
2869199514,39cyclictest0-21swapper/321:34:063
2869199514,39cyclictest0-21swapper/321:34:063
2869199514,29cyclictest0-21swapper/319:33:493
2869199514,28cyclictest0-21swapper/320:59:173
2869199514,28cyclictest0-21swapper/318:49:113
2869199514,24cyclictest0-21swapper/321:58:593
2869199514,12cyclictest0-21swapper/320:15:143
28691995120,12cyclictest0-21swapper/317:49:063
2869099515,38cyclictest6448-21df22:33:582
2869099514,29cyclictest0-21swapper/220:54:122
2869099513,30cyclictest0-21swapper/219:04:022
2868999514,38cyclictest13784-21unixbench_multi19:24:241
2868999514,38cyclictest13784-21unixbench_multi19:24:241
2868999514,29cyclictest0-21swapper/121:09:161
2868999514,29cyclictest0-21swapper/117:54:051
2868999514,27cyclictest0-21swapper/121:33:591
2868999514,27cyclictest0-21swapper/121:33:591
2868999513,27cyclictest0-21swapper/120:09:211
2868999513,10cyclictest0-21swapper/120:59:211
28689995122,11cyclictest1265-21runrttasks18:04:261
223099510,5rtkit-daemon2229-21rtkit-daemon22:53:553
223099510,5rtkit-daemon2229-21rtkit-daemon22:53:553
56992503,41sleep10-21swapper/118:00:561
56992503,41sleep10-21swapper/118:00:561
297032502,41sleep30-21swapper/317:44:133
2869199504,39cyclictest0-21swapper/318:19:233
2869199504,28cyclictest0-21swapper/322:24:143
2869199504,27cyclictest0-21swapper/321:24:033
2869199504,27cyclictest0-21swapper/319:44:253
28691995033,10cyclictest30148-21aten_r1power_fr18:54:043
2869199503,13cyclictest0-21swapper/318:39:053
2869099504,28cyclictest0-21swapper/223:08:562
2869099504,28cyclictest0-21swapper/220:44:142
2869099504,28cyclictest0-21swapper/220:09:132
2869099504,28cyclictest0-21swapper/220:04:082
2869099504,28cyclictest0-21swapper/220:04:082
2869099504,28cyclictest0-21swapper/218:29:102
2869099504,28cyclictest0-21swapper/218:14:042
2869099504,27cyclictest0-21swapper/218:34:192
28690995034,10cyclictest4079-21tr17:59:082
28690995034,10cyclictest4079-21tr17:59:082
28690995033,10cyclictest13737-21df22:48:572
28690995011,21cyclictest0-21swapper/219:24:132
28690995011,21cyclictest0-21swapper/219:24:132
2868999505,29cyclictest0-21swapper/123:08:541
2868999505,28cyclictest0-21swapper/118:49:101
2868999505,27cyclictest0-21swapper/120:29:001
2868999504,39cyclictest0-21swapper/122:23:571
2868999504,30cyclictest0-21swapper/122:29:121
2868999504,27cyclictest0-21swapper/118:44:101
2868999504,27cyclictest0-21swapper/118:44:101
2868999504,27cyclictest0-21swapper/118:34:281
28689995021,12cyclictest0-21swapper/120:19:101
28689995019,12cyclictest0-21swapper/121:39:211
28689995018,14cyclictest0-21swapper/117:49:221
174422504,40sleep10-21swapper/122:54:551
174422504,40sleep10-21swapper/122:54:551
115392503,11sleep10-21swapper/119:19:481
9052493,40sleep10-21swapper/120:08:411
9052493,40sleep10-21swapper/120:08:411
85072493,7sleep38541-21open_files21:29:123
85072493,7sleep38541-21open_files21:29:123
81132493,10sleep10-21swapper/122:37:421
60892494,9sleep26117-21telnet20:18:582
2869199494,38cyclictest0-21swapper/318:09:223
2869199494,28cyclictest0-21swapper/319:54:013
2869199494,27cyclictest0-21swapper/322:38:423
2869199494,27cyclictest0-21swapper/321:14:103
2869199494,27cyclictest0-21swapper/320:29:023
28691994932,10cyclictest14928-21sed20:34:233
2869199493,28cyclictest0-21swapper/319:39:133
2869199493,24cyclictest0-21swapper/320:48:463
2869099494,28cyclictest0-21swapper/218:44:052
2869099494,28cyclictest0-21swapper/218:44:052
2869099494,27cyclictest0-21swapper/223:04:022
2869099494,27cyclictest0-21swapper/222:43:552
2869099494,27cyclictest0-21swapper/222:23:432
2869099494,27cyclictest0-21swapper/221:44:222
2869099494,27cyclictest0-21swapper/220:24:262
2869099494,26cyclictest0-21swapper/220:53:472
2869099494,26cyclictest0-21swapper/220:53:472
28690994935,8cyclictest17715-21grep19:34:102
28690994935,8cyclictest1658-21unixbench_multi21:14:242
2868999495,26cyclictest0-21swapper/121:15:221
2868999494,28cyclictest0-21swapper/122:49:131
2868999494,28cyclictest0-21swapper/122:40:361
2868999494,28cyclictest0-21swapper/122:08:561
2868999494,28cyclictest0-21swapper/119:44:131
2868999494,27cyclictest0-21swapper/121:48:591
2868999493,28cyclictest0-21swapper/121:29:141
2868999493,28cyclictest0-21swapper/121:29:141
28689994918,13cyclictest0-21swapper/123:04:071
28689994918,13cyclictest0-21swapper/122:59:061
28689994914,13cyclictest0-21swapper/122:43:541
228072492,10sleep30-21swapper/320:53:493
223099490,5rtkit-daemon2229-21rtkit-daemon20:15:382
2869199484,27cyclictest0-21swapper/320:05:123
2869199484,27cyclictest0-21swapper/320:05:123
2869199484,27cyclictest0-21swapper/319:19:133
2869199484,26cyclictest0-21swapper/321:54:133
28691994832,10cyclictest2901-21cpuspeed_turbos19:04:043
28691994831,11cyclictest16509-21gltestperf20:39:083
28691994831,10cyclictest27851-21switchtime22:09:163
28691994818,12cyclictest0-21swapper/318:48:523
2869099484,27cyclictest0-21swapper/218:53:512
2869099484,24cyclictest20121-21sadc23:03:412
2869099484,24cyclictest0-21swapper/217:54:052
2869099484,24cyclictest0-21swapper/217:44:062
2869099483,27cyclictest0-21swapper/218:24:202
28690994829,11cyclictest2832-21grep21:19:002
2868999484,28cyclictest0-21swapper/119:03:501
2868999484,23cyclictest0-21swapper/118:39:051
28689994820,11cyclictest0-21swapper/118:09:541
28689994818,13cyclictest0-21swapper/120:54:021
283992484,36sleep028389-21yum-updatesd-he17:40:370
2869199474,36cyclictest0-21swapper/322:15:323
2869199474,36cyclictest0-21swapper/322:15:323
2869199472,38cyclictest0-21swapper/322:43:493
2869099475,25cyclictest0-21swapper/220:29:002
2869099474,36cyclictest0-21swapper/218:59:102
2869099474,26cyclictest0-21swapper/220:59:102
28690994717,13cyclictest0-21swapper/219:44:192
28689994727,13cyclictest0-21swapper/120:49:091
28689994727,13cyclictest0-21swapper/120:49:091
223099470,5rtkit-daemon2229-21rtkit-daemon21:06:063
2869199464,34cyclictest0-21swapper/321:48:453
28691994630,10cyclictest7936-21unixbench_singl18:04:283
2869099464,34cyclictest0-21swapper/221:49:112
2869099464,26cyclictest0-21swapper/222:14:212
2869099464,26cyclictest0-21swapper/222:14:212
2869099464,26cyclictest0-21swapper/219:14:232
28690994632,9cyclictest16258-21cut20:39:062
2869099463,26cyclictest0-21swapper/221:34:082
2869099463,26cyclictest0-21swapper/221:34:082
28690994629,11cyclictest9258-21gltestperf22:39:022
28689994627,12cyclictest28684-21wc21:04:131
138502463,10sleep20-21swapper/221:39:272
92292452,9sleep20-21swapper/221:31:412
92292452,9sleep20-21swapper/221:31:412
307332452,38sleep20-21swapper/220:00:402
28691994532,8cyclictest25307-21aten_r1power_cu18:44:063
28691994530,9cyclictest8486-21munin-node20:24:053
2869199453,25cyclictest0-21swapper/319:00:023
2869199453,12cyclictest0-21swapper/319:59:113
28691994529,10cyclictest6968-21gltestperf20:19:063
28690994531,8cyclictest31449-21aten2.4-expect17:49:072
2868999454,22cyclictest0-21swapper/118:29:061
247042452,9sleep30-21swapper/323:11:183
28691994420,19cyclictest1265-21runrttasks21:19:283
2869099444,34cyclictest0-21swapper/221:28:442
2869099444,32cyclictest0-21swapper/219:49:192
28690994423,14cyclictest0-21swapper/222:04:032
2868999443,10cyclictest14688-21cpuspeed21:44:031
2869099434,24cyclictest0-21swapper/222:27:122
28690994329,8cyclictest14130-21gltestperf20:34:092
2868999377,21cyclictest0-21swapper/117:43:531
270022349,15sleep30-21swapper/317:39:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional