You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-17 - 05:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Oct 17, 2025 00:45:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
245878722560,8sleep00-21swapper/019:05:140
245954422400,8sleep12197014-21PK-Backend19:05:271
255351821320,13sleep3411rcuc/323:40:153
1623991080,26rtkit-daemon921-21thermald20:09:330
247285221010,6sleep2331rcuc/219:45:252
24671102940,8sleep31753-21gnome-shell19:30:163
162399870,75rtkit-daemon0-21swapper/319:06:233
162399820,10rtkit-daemon1753-21gnome-shell21:22:092
162399820,10rtkit-daemon1753-21gnome-shell21:22:092
162399760,6rtkit-daemon0-21swapper/221:42:302
162399760,6rtkit-daemon0-21swapper/221:42:302
162399750,6rtkit-daemon0-21swapper/223:08:482
245915127458,10sleep20-21swapper/219:05:222
162399740,5rtkit-daemon0-21swapper/221:31:152
162399740,10rtkit-daemon1622-21rtkit-daemon00:13:011
162399720,17rtkit-daemon1622-21rtkit-daemon22:26:210
162399690,6rtkit-daemon1622-21rtkit-daemon23:47:281
246024999668,33cyclictest2533408-21cat22:45:231
162399660,9rtkit-daemon1622-21rtkit-daemon22:47:160
162399660,19rtkit-daemon1622-21rtkit-daemon21:27:160
162399660,18rtkit-daemon1622-21rtkit-daemon22:15:040
162399650,18rtkit-daemon1622-21rtkit-daemon23:18:220
246024999646,31cyclictest2563634-21date00:10:021
162399640,10rtkit-daemon1622-21rtkit-daemon23:38:470
2460249996313,25cyclictest2482876-21cut20:15:101
2460249996313,25cyclictest2482876-21cut20:15:101
2460249996310,36cyclictest2466706-21seq19:28:371
25353812620,5chrt0-21swapper/122:50:261
246024999627,37cyclictest1753-21gnome-shell23:35:011
2460249996211,29cyclictest1753-21gnome-shell22:25:001
2460249996210,17cyclictest1753-21gnome-shell21:52:411
162399620,10rtkit-daemon1622-21rtkit-daemon23:28:120
162399620,10rtkit-daemon1622-21rtkit-daemon23:28:120
25046922600,6sleep3411rcuc/321:15:333
25046922600,6sleep3411rcuc/321:15:333
246024999608,19cyclictest1753-21gnome-shell21:06:251
162399600,9rtkit-daemon1622-21rtkit-daemon21:59:240
162399600,9rtkit-daemon1622-21rtkit-daemon19:57:391
162399600,8rtkit-daemon1622-21rtkit-daemon21:02:010
162399600,8rtkit-daemon1622-21rtkit-daemon21:02:000
162399600,8rtkit-daemon1622-21rtkit-daemon19:56:310
246024999598,17cyclictest1753-21gnome-shell22:08:201
246024999598,16cyclictest1753-21gnome-shell00:38:321
246024999597,29cyclictest2541319-40lsb_release23:06:201
246024999597,17cyclictest1753-21gnome-shell19:14:051
246024999597,17cyclictest1753-21gnome-shell19:14:051
246024999597,16cyclictest1753-21gnome-shell23:35:021
2460249995910,15cyclictest2488173-40apt-check20:30:181
162399590,9rtkit-daemon1622-21rtkit-daemon19:16:170
162399590,9rtkit-daemon1622-21rtkit-daemon19:16:170
162399590,8rtkit-daemon1622-21rtkit-daemon22:17:161
246024999588,16cyclictest1753-21gnome-shell21:17:481
246024999588,16cyclictest1753-21gnome-shell21:17:471
246024999588,16cyclictest1753-21gnome-shell20:50:471
246024999587,30cyclictest2502950-21/usr/sbin/munin21:10:281
246024999587,26cyclictest1753-21gnome-shell20:21:431
246024999587,16cyclictest1753-21gnome-shell20:13:551
2460249995813,17cyclictest1753-21gnome-shell21:40:101
2460249995813,17cyclictest1753-21gnome-shell21:40:091
162399580,9rtkit-daemon1622-21rtkit-daemon23:29:071
162399580,9rtkit-daemon1622-21rtkit-daemon23:29:071
162399580,8rtkit-daemon1622-21rtkit-daemon19:30:351
162399580,5rtkit-daemon1753-21gnome-shell19:29:420
246024999579,26cyclictest2487662-21http20:30:001
246024999579,26cyclictest2487662-21http20:30:001
2460249995713,19cyclictest2566823-21seq00:18:561
162399570,8rtkit-daemon1622-21rtkit-daemon23:09:500
24698002560,7sleep32197014-21PK-Backend19:35:323
246024999569,11cyclictest1753-21gnome-shell19:50:101
246024999567,15cyclictest1753-21gnome-shell00:25:161
2460249995614,18cyclictest2554483-40apt-check23:42:231
2460249995611,15cyclictest2539033-21/usr/sbin/munin23:00:311
2460249995610,18cyclictest2479323-21cpuspeed_turbos20:05:091
162399560,9rtkit-daemon1622-21rtkit-daemon22:11:221
162399560,8rtkit-daemon1622-21rtkit-daemon23:17:321
25485672550,6sleep2331rcuc/223:25:292
25485672550,6sleep2331rcuc/223:25:292
25174172550,18sleep32197014-21PK-Backend22:00:183
25174172550,18sleep32197014-21PK-Backend22:00:183
246024999557,15cyclictest1753-21gnome-shell21:27:291
246024999556,15cyclictest1753-21gnome-shell20:43:031
246024999555,17cyclictest1753-21gnome-shell22:33:211
246024999555,16cyclictest1753-21gnome-shell22:37:221
2460249995515,11cyclictest1753-21gnome-shell19:40:331
2460249995514,21cyclictest2516539-21seq21:57:091
2460249995511,15cyclictest2498600-21basename21:00:121
2460249995511,15cyclictest2498600-21basename21:00:121
162399550,9rtkit-daemon1622-21rtkit-daemon20:04:331
246024999549,17cyclictest1753-21gnome-shell23:53:251
246024999548,17cyclictest1753-21gnome-shell19:17:491
246024999548,17cyclictest1753-21gnome-shell19:17:491
246024999546,15cyclictest1753-21gnome-shell21:37:141
246024999545,31cyclictest2542490-21cat23:10:141
2460249995413,24cyclictest1753-21gnome-shell22:26:571
2460249995411,11cyclictest1753-21gnome-shell00:23:061
2460249995410,24cyclictest2468802-21seq19:35:041
2460246995425,16cyclictest2468311-21taskset19:31:590
162399540,8rtkit-daemon1622-21rtkit-daemon20:57:171
162399540,8rtkit-daemon1622-21rtkit-daemon20:00:540
246024999539,17cyclictest1753-21gnome-shell19:23:581
2460249995311,15cyclictest2561463-21taskset00:01:521
2460249995310,19cyclictest2571050-21/usr/sbin/munin00:30:191
2460249995310,19cyclictest2571050-21/usr/sbin/munin00:30:191
2460249995310,19cyclictest2518269-21runrttasks22:02:061
2460249995310,19cyclictest2518269-21runrttasks22:02:061
2460249995310,16cyclictest2490537-21memory20:35:281
2460246995325,15cyclictest2532201-21runrttasks22:42:530
2460254995216,21cyclictest1753-21gnome-shell19:49:043
246024999529,26cyclictest1753-21gnome-shell20:45:261
246024999529,19cyclictest2505878-40lsb_release21:21:201
246024999529,19cyclictest2505878-40lsb_release21:21:191
246024999528,17cyclictest1753-21gnome-shell23:58:021
246024999527,16cyclictest1753-21gnome-shell22:55:201
2460249995211,24cyclictest1753-21gnome-shell21:45:451
2460249995210,19cyclictest2473688-21runrttasks19:49:121
162399520,45rtkit-daemon1775-21gvfs-afc-volume20:37:480
246024999517,26cyclictest1753-21gnome-shell23:20:061
246024999517,26cyclictest1753-21gnome-shell23:20:061
2460249995112,15cyclictest1753-21gnome-shell22:44:211
2460246995127,15cyclictest13-21ksoftirqd/022:30:160
2460246995125,15cyclictest2521269-21uname22:10:270
162399510,9rtkit-daemon1753-21gnome-shell20:52:140
162399510,43rtkit-daemon1506-21systemd00:32:420
162399510,43rtkit-daemon1506-21systemd00:32:420
24610482500,5chrt0-21swapper/219:10:232
24610482500,5chrt0-21swapper/219:10:222
246025199508,6cyclictest141rcu_preempt23:40:042
246025199507,6cyclictest141rcu_preempt23:17:472
2460251995024,17cyclictest34-21ksoftirqd/223:12:222
2460251995023,16cyclictest34-21ksoftirqd/221:55:152
2460251995015,10cyclictest34-21ksoftirqd/219:20:152
2460246995026,14cyclictest13-21ksoftirqd/022:20:140
2460246995011,19cyclictest2414695-21kworker/0:020:15:070
2460246995011,19cyclictest2414695-21kworker/0:020:15:070
2460246995011,17cyclictest2414695-21kworker/0:020:45:220
162399500,43rtkit-daemon1506-21systemd23:03:400
246025199498,20cyclictest34-21ksoftirqd/221:50:032
246025199491,5cyclictest141rcu_preempt19:58:572
2460246994928,17cyclictest2488592-40lsb_release20:30:240
2460246994926,14cyclictest13-21ksoftirqd/000:09:410
2460246994925,15cyclictest13-21ksoftirqd/023:45:290
2460246994923,22cyclictest2554501-40lsb_release23:42:200
2460246994920,20cyclictest13-21ksoftirqd/020:10:310
162399490,41rtkit-daemon1506-21systemd23:51:120
2460254994829,5cyclictest2496605-21cron20:55:003
2460251994832,6cyclictest34-21ksoftirqd/223:34:592
2460246994837,8cyclictest13-21ksoftirqd/021:40:020
2460246994825,6cyclictest2561020-21ntpq00:00:300
2460246994824,14cyclictest13-21ksoftirqd/019:20:180
25658322470,5sleep3411rcuc/300:15:173
2460254994731,12cyclictest855-21dbus-daemon21:55:003
2460251994730,6cyclictest34-21ksoftirqd/221:50:002
2460251994729,6cyclictest34-21ksoftirqd/222:29:192
2460246994728,15cyclictest13-21ksoftirqd/000:23:180
2460246994725,18cyclictest13-21ksoftirqd/021:40:180
2460246994725,18cyclictest13-21ksoftirqd/021:40:180
2460254994615,15cyclictest141rcu_preempt21:01:363
2460254994615,15cyclictest141rcu_preempt21:01:363
2460251994625,16cyclictest34-21ksoftirqd/221:00:252
2460251994625,16cyclictest34-21ksoftirqd/221:00:252
2460251994622,13cyclictest34-21ksoftirqd/223:49:472
2460251994614,10cyclictest34-21ksoftirqd/200:30:012
2460246994627,15cyclictest13-21ksoftirqd/019:14:160
2460246994627,15cyclictest13-21ksoftirqd/019:14:150
2460246994625,16cyclictest13-21ksoftirqd/023:10:230
2460246994624,13cyclictest13-21ksoftirqd/022:00:290
2460246994624,13cyclictest13-21ksoftirqd/022:00:290
24612892450,7sleep32197014-21PK-Backend19:10:273
24612892450,7sleep32197014-21PK-Backend19:10:273
2460254994523,13cyclictest42-21ksoftirqd/322:45:233
2460254994518,10cyclictest1753-21gnome-shell21:10:473
246025199456,18cyclictest34-21ksoftirqd/220:05:002
2460251994525,6cyclictest34-21ksoftirqd/222:35:442
2460251994524,7cyclictest34-21ksoftirqd/200:35:212
2460251994523,14cyclictest34-21ksoftirqd/220:40:122
2460251994522,14cyclictest34-21ksoftirqd/221:05:322
2460251994521,14cyclictest34-21ksoftirqd/222:55:032
2460251994520,12cyclictest34-21ksoftirqd/222:45:222
246025199451,15cyclictest2481923-21chrt20:10:332
246024999458,15cyclictest2508602-21munin-run21:35:001
2460246994528,6cyclictest13-21ksoftirqd/023:56:360
2460246994526,14cyclictest13-21ksoftirqd/022:50:130
2460246994526,14cyclictest13-21ksoftirqd/000:17:500
2460246994525,15cyclictest13-21ksoftirqd/020:55:190
2460246994525,15cyclictest13-21ksoftirqd/000:10:350
2460246994522,19cyclictest13-21ksoftirqd/021:25:030
2460246994522,19cyclictest13-21ksoftirqd/021:25:020
25469442440,5chrt411rcuc/323:20:323
25469442440,5chrt411rcuc/323:20:313
2460254994419,7cyclictest1753-21gnome-shell19:44:473
2460254994418,21cyclictest42-21ksoftirqd/322:15:203
2460251994426,7cyclictest1753-21gnome-shell23:35:132
2460251994426,6cyclictest1753-21gnome-shell00:02:152
2460251994425,7cyclictest34-21ksoftirqd/219:40:172
2460251994425,6cyclictest34-21ksoftirqd/221:25:152
2460251994424,6cyclictest34-21ksoftirqd/200:20:272
2460251994423,6cyclictest34-21ksoftirqd/200:19:012
2460251994423,6cyclictest34-21ksoftirqd/200:10:552
2460251994421,14cyclictest34-21ksoftirqd/223:50:292
2460251994420,14cyclictest34-21ksoftirqd/222:52:552
2460246994425,6cyclictest13-21ksoftirqd/020:40:310
2460246994425,15cyclictest13-21ksoftirqd/023:20:320
2460246994425,15cyclictest13-21ksoftirqd/023:20:320
2460246994425,14cyclictest13-21ksoftirqd/022:06:490
2460246994425,14cyclictest13-21ksoftirqd/021:15:220
2460246994425,14cyclictest13-21ksoftirqd/021:15:220
2460246994425,14cyclictest13-21ksoftirqd/021:10:180
2460246994425,14cyclictest13-21ksoftirqd/019:50:000
2460246994425,14cyclictest13-21ksoftirqd/019:35:170
2460246994425,14cyclictest13-21ksoftirqd/000:26:480
2460246994424,15cyclictest13-21ksoftirqd/021:54:530
2460246994423,16cyclictest13-21ksoftirqd/020:25:340
2460246994423,16cyclictest13-21ksoftirqd/020:25:330
246025499439,14cyclictest1775-21gvfs-afc-volume22:31:553
2460254994332,8cyclictest1753-21gnome-shell20:05:283
246025499433,25cyclictest141rcu_preempt00:31:113
246025499433,25cyclictest141rcu_preempt00:31:103
2460254994324,9cyclictest42-21ksoftirqd/323:18:503
2460254994321,18cyclictest2562750-21ntp_states00:05:313
2460254994313,6cyclictest0-21swapper/320:03:113
2460254994313,18cyclictest42-21ksoftirqd/322:06:003
2460251994325,6cyclictest34-21ksoftirqd/221:35:372
2460251994325,6cyclictest34-21ksoftirqd/219:19:482
2460251994325,6cyclictest34-21ksoftirqd/219:19:472
2460251994324,6cyclictest34-21ksoftirqd/220:45:152
2460251994324,6cyclictest34-21ksoftirqd/220:35:242
2460251994324,6cyclictest34-21ksoftirqd/219:25:302
2460251994324,5cyclictest34-21ksoftirqd/219:37:152
2460251994323,6cyclictest34-21ksoftirqd/200:35:022
2460251994323,6cyclictest34-21ksoftirqd/200:35:012
2460251994321,13cyclictest34-21ksoftirqd/223:56:132
2460251994321,13cyclictest34-21ksoftirqd/220:55:082
2460251994321,13cyclictest34-21ksoftirqd/200:06:362
2460251994321,12cyclictest34-21ksoftirqd/220:52:362
2460251994319,13cyclictest34-21ksoftirqd/222:12:452
246025199431,21cyclictest34-21ksoftirqd/222:15:202
246024699434,34cyclictest2508588-21perf21:35:000
2460246994325,14cyclictest13-21ksoftirqd/022:59:450
2460246994325,14cyclictest13-21ksoftirqd/000:37:380
2460246994324,14cyclictest13-21ksoftirqd/023:30:200
2460246994324,14cyclictest13-21ksoftirqd/021:05:280
2460246994324,14cyclictest13-21ksoftirqd/020:25:010
2460246994324,14cyclictest13-21ksoftirqd/019:50:230
2460246994323,15cyclictest13-21ksoftirqd/021:49:380
2460246994322,13cyclictest13-21ksoftirqd/019:40:140
246025499429,20cyclictest42-21ksoftirqd/320:10:043
246025499424,25cyclictest141rcu_preempt23:35:293
2460254994224,14cyclictest42-21ksoftirqd/321:30:013
2460254994223,14cyclictest42-21ksoftirqd/320:30:243
2460251994224,6cyclictest34-21ksoftirqd/220:30:172
2460251994223,6cyclictest34-21ksoftirqd/223:00:172
2460251994223,6cyclictest34-21ksoftirqd/221:15:312
2460251994223,6cyclictest34-21ksoftirqd/221:15:312
2460251994223,6cyclictest34-21ksoftirqd/221:10:192
2460251994222,5cyclictest34-21ksoftirqd/222:05:152
2460251994220,14cyclictest34-21ksoftirqd/220:18:312
2460251994220,14cyclictest34-21ksoftirqd/220:18:302
2460251994220,14cyclictest34-21ksoftirqd/220:05:162
2460251994220,13cyclictest34-21ksoftirqd/219:50:102
2460246994224,14cyclictest13-21ksoftirqd/022:39:160
246025499417,17cyclictest1753-21gnome-shell21:31:523
2460254994130,7cyclictest1753-21gnome-shell19:25:233
2460254994127,9cyclictest1753-21gnome-shell22:27:283
2460254994124,13cyclictest42-21ksoftirqd/323:46:343
2460254994124,13cyclictest42-21ksoftirqd/323:35:013
2460254994124,13cyclictest42-21ksoftirqd/300:21:383
2460254994123,6cyclictest42-21ksoftirqd/321:20:053
2460254994123,6cyclictest42-21ksoftirqd/321:20:043
2460254994123,14cyclictest42-21ksoftirqd/323:00:353
2460254994123,14cyclictest42-21ksoftirqd/322:50:143
2460254994123,14cyclictest42-21ksoftirqd/319:16:373
2460254994123,14cyclictest42-21ksoftirqd/319:16:373
2460254994122,14cyclictest42-21ksoftirqd/322:20:243
2460254994122,14cyclictest42-21ksoftirqd/320:40:323
2460254994122,14cyclictest42-21ksoftirqd/320:15:223
2460254994122,14cyclictest42-21ksoftirqd/320:15:223
246025499411,6cyclictest0-21swapper/319:51:073
2460254994116,21cyclictest42-21ksoftirqd/321:35:273
246025499411,5cyclictest2197014-21PK-Backend20:35:493
2460254994114,7cyclictest42-21ksoftirqd/320:29:133
2460254994114,7cyclictest42-21ksoftirqd/320:29:133
2460254994113,7cyclictest2569504-21cat00:25:253
2460251994123,6cyclictest34-21ksoftirqd/222:00:292
2460251994123,6cyclictest34-21ksoftirqd/222:00:292
2460251994123,6cyclictest34-21ksoftirqd/219:30:022
2460251994121,6cyclictest34-21ksoftirqd/220:21:062
2460251994119,14cyclictest34-21ksoftirqd/223:21:172
2460251994119,14cyclictest34-21ksoftirqd/223:21:162
2460251994119,13cyclictest34-21ksoftirqd/222:40:322
25250342400,6sleep2331rcuc/222:20:322
2460254994023,13cyclictest42-21ksoftirqd/323:05:033
2460251994023,6cyclictest34-21ksoftirqd/220:27:142
2460251994023,6cyclictest34-21ksoftirqd/220:27:132
2460251994022,6cyclictest34-21ksoftirqd/222:30:232
2460254993922,13cyclictest42-21ksoftirqd/321:41:563
2460254993922,13cyclictest42-21ksoftirqd/321:41:553
2460254993922,13cyclictest42-21ksoftirqd/319:56:573
2460254993921,14cyclictest42-21ksoftirqd/300:00:123
2460254993920,6cyclictest42-21ksoftirqd/322:10:243
2460254993920,6cyclictest42-21ksoftirqd/300:00:003
2460254993920,14cyclictest42-21ksoftirqd/320:20:213
246025499391,15cyclictest860997-21snmpd20:45:203
2460254993910,6cyclictest141rcu_preempt19:24:593
2460254993822,12cyclictest42-21ksoftirqd/300:35:263
2460254993822,12cyclictest2557450-21netstat23:50:253
2460254993820,6cyclictest42-21ksoftirqd/321:05:163
2460254993820,14cyclictest42-21ksoftirqd/322:40:013
2460254993820,13cyclictest42-21ksoftirqd/322:40:353
2460254993819,6cyclictest42-21ksoftirqd/320:55:323
2460254993819,6cyclictest42-21ksoftirqd/300:14:083
246025499375,17cyclictest1753-21gnome-shell22:56:563
2460254993719,14cyclictest42-21ksoftirqd/323:14:193
2460254993711,9cyclictest2197014-21PK-Backend21:48:213
2460254993623,10cyclictest42-21ksoftirqd/323:28:223
2460254993623,10cyclictest42-21ksoftirqd/323:28:223
2460254993622,10cyclictest42-21ksoftirqd/321:55:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional