You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 07:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Wed Dec 03, 2025 00:44:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
414580222430,6sleep1251rcuc/119:05:291
414580222430,6sleep1251rcuc/119:05:281
1666221070,5sleep11753-21gnome-shell22:10:011
1666221070,5sleep11753-21gnome-shell22:10:011
1623991060,8rtkit-daemon1622-21rtkit-daemon23:56:362
1905821020,7chrt2197014-21PK-Backend22:15:222
1623991000,8rtkit-daemon1622-21rtkit-daemon19:21:272
1623991000,8rtkit-daemon1622-21rtkit-daemon19:21:262
162399980,9rtkit-daemon1622-21rtkit-daemon00:31:442
162399970,9rtkit-daemon1622-21rtkit-daemon23:45:332
162399970,9rtkit-daemon1622-21rtkit-daemon22:20:352
162399950,8rtkit-daemon1622-21rtkit-daemon21:59:042
162399910,8rtkit-daemon1622-21rtkit-daemon22:59:352
162399900,8rtkit-daemon1622-21rtkit-daemon22:46:382
162399880,9rtkit-daemon1622-21rtkit-daemon19:44:372
162399880,9rtkit-daemon1622-21rtkit-daemon19:44:362
162399860,9rtkit-daemon1622-21rtkit-daemon19:10:352
418082840,5sleep22197014-21PK-Backend23:20:142
162399840,8rtkit-daemon1622-21rtkit-daemon23:52:002
162399840,8rtkit-daemon1622-21rtkit-daemon22:38:152
162399760,67rtkit-daemon0-21swapper/319:05:253
162399760,67rtkit-daemon0-21swapper/319:05:243
414622727039,24sleep00-21swapper/019:08:050
414622727039,24sleep00-21swapper/019:08:050
414651899694,43cyclictest1753-21gnome-shell21:05:203
414651899656,36cyclictest1753-21gnome-shell21:37:003
414651599651,10cyclictest4160414-21ntp_states19:45:252
4146515996436,10cyclictest34-21ksoftirqd/219:17:332
414651599640,60cyclictest1622-21rtkit-daemon23:00:152
414608426446,13sleep20-21swapper/219:06:192
414608426446,13sleep20-21swapper/219:06:192
414651899635,18cyclictest1753-21gnome-shell00:25:163
414650499637,20cyclictest1753-21gnome-shell21:12:210
558092620,6sleep22197014-21PK-Backend00:00:192
414651899624,36cyclictest1753-21gnome-shell19:17:293
4146509996231,16cyclictest26-21ksoftirqd/123:26:531
162399620,9rtkit-daemon1622-21rtkit-daemon21:47:162
510552610,7sleep12197014-21PK-Backend23:45:291
414651899616,31cyclictest1753-21gnome-shell20:36:583
414651899616,30cyclictest1753-21gnome-shell22:27:033
414651899616,28cyclictest1753-21gnome-shell23:00:023
414651899615,28cyclictest1753-21gnome-shell19:20:263
414651899615,28cyclictest1753-21gnome-shell19:20:263
4146515996136,10cyclictest34-21ksoftirqd/222:07:022
4146515996136,10cyclictest34-21ksoftirqd/222:07:012
4146515996134,10cyclictest34-21ksoftirqd/222:01:062
414650499611,16cyclictest4150276-21chrt19:18:310
414651899608,30cyclictest1753-21gnome-shell21:10:183
414651899607,29cyclictest1753-21gnome-shell00:35:263
414651899607,27cyclictest1753-21gnome-shell22:47:303
414651899607,27cyclictest1753-21gnome-shell22:43:293
414651899607,27cyclictest1753-21gnome-shell21:52:063
414651899606,29cyclictest1753-21gnome-shell20:30:293
414651899606,29cyclictest1753-21gnome-shell00:07:113
414651899606,28cyclictest1753-21gnome-shell23:47:093
414651899606,28cyclictest1753-21gnome-shell22:01:133
414651899606,28cyclictest1753-21gnome-shell20:27:593
414651899606,27cyclictest1753-21gnome-shell22:10:343
414651899606,27cyclictest1753-21gnome-shell20:17:133
414651899606,25cyclictest1753-21gnome-shell23:31:163
414651899606,25cyclictest1753-21gnome-shell23:31:163
414651899605,32cyclictest1753-21gnome-shell00:15:013
414651899605,28cyclictest1753-21gnome-shell19:54:443
414651899605,28cyclictest1753-21gnome-shell00:21:303
414651899604,29cyclictest1753-21gnome-shell23:40:233
414651899596,30cyclictest1753-21gnome-shell22:20:233
414651899596,29cyclictest1753-21gnome-shell00:15:033
414651899596,28cyclictest1753-21gnome-shell23:35:153
414651899596,28cyclictest1753-21gnome-shell21:48:063
414651899596,28cyclictest1753-21gnome-shell21:00:383
414651899596,28cyclictest1753-21gnome-shell20:45:373
414651899596,27cyclictest1753-21gnome-shell23:21:403
414651899596,27cyclictest1753-21gnome-shell23:18:133
414651899596,27cyclictest1753-21gnome-shell22:56:063
414651899596,27cyclictest1753-21gnome-shell22:07:023
414651899596,27cyclictest1753-21gnome-shell22:07:013
414651899596,27cyclictest1753-21gnome-shell21:34:043
414651899596,27cyclictest1753-21gnome-shell20:56:383
414651899596,27cyclictest1753-21gnome-shell20:00:103
414651899596,26cyclictest1753-21gnome-shell20:05:213
414651899596,26cyclictest1753-21gnome-shell19:45:493
414651899595,28cyclictest1753-21gnome-shell23:25:413
414651899595,28cyclictest1753-21gnome-shell23:05:323
414651899595,28cyclictest1753-21gnome-shell22:35:343
414651899595,27cyclictest1753-21gnome-shell20:40:173
414651899595,25cyclictest1753-21gnome-shell22:15:123
414651899595,17cyclictest1753-21gnome-shell00:00:223
414651899594,28cyclictest1753-21gnome-shell19:25:023
4146515995930,7cyclictest34-21ksoftirqd/220:30:242
414651899586,28cyclictest1753-21gnome-shell19:30:093
414651899585,28cyclictest1753-21gnome-shell23:12:363
414651899585,28cyclictest1753-21gnome-shell22:34:163
414651899585,28cyclictest1753-21gnome-shell22:34:153
414651899585,27cyclictest1753-21gnome-shell23:51:073
414651899585,27cyclictest1753-21gnome-shell00:34:423
414651599585,38cyclictest0-21swapper/223:09:112
414650999580,54cyclictest1622-21rtkit-daemon20:17:371
41585232570,8sleep01753-21gnome-shell19:40:250
41585232570,8sleep01753-21gnome-shell19:40:240
414651899575,27cyclictest1753-21gnome-shell21:59:033
414651899574,28cyclictest1753-21gnome-shell20:10:033
414651899574,27cyclictest1753-21gnome-shell19:55:283
414651599572,17cyclictest1753-21gnome-shell00:25:262
4146504995710,18cyclictest1753-21gnome-shell20:15:290
414651899553,28cyclictest1753-21gnome-shell20:20:083
4146515995543,9cyclictest34-21ksoftirqd/223:15:292
4146509995518,16cyclictest26-21ksoftirqd/123:31:031
4146509995518,16cyclictest26-21ksoftirqd/123:31:031
4146504995510,9cyclictest52280-21df23:50:180
162399550,8rtkit-daemon1622-21rtkit-daemon00:36:162
4146515995435,5cyclictest34-21ksoftirqd/221:36:292
4146515995412,23cyclictest34-21ksoftirqd/219:59:552
4146515995410,14cyclictest34-21ksoftirqd/222:42:412
414650999548,31cyclictest26-21ksoftirqd/119:20:071
414650999548,31cyclictest26-21ksoftirqd/119:20:071
4146509995444,7cyclictest251rcuc/120:55:081
4146509995414,12cyclictest26-21ksoftirqd/120:50:311
414650499549,17cyclictest1753-21gnome-shell19:28:410
414650499549,17cyclictest1753-21gnome-shell00:02:470
414650499542,16cyclictest1753-21gnome-shell20:30:160
414650499542,15cyclictest20747-21hddtemp_smartct22:20:200
414650499541,13cyclictest4187451-21cron21:05:000
4146515995331,9cyclictest34-21ksoftirqd/200:08:162
4146509995344,6cyclictest251rcuc/120:30:201
4146509995315,16cyclictest1938-21gsd-housekeepin19:17:001
414650499539,17cyclictest1753-21gnome-shell23:04:520
414650499536,16cyclictest69273-21seq00:36:120
414650499536,16cyclictest1753-21gnome-shell00:12:020
4146504995310,17cyclictest1753-21gnome-shell21:19:420
4146515995230,10cyclictest34-21ksoftirqd/221:01:452
4146515995223,9cyclictest34-21ksoftirqd/200:15:002
414651599521,6cyclictest141rcu_preempt23:30:322
414651599521,6cyclictest141rcu_preempt23:30:322
414650999526,10cyclictest26-21ksoftirqd/123:58:161
4146509995244,5cyclictest251rcuc/120:01:441
4146509995243,6cyclictest251rcuc/121:23:201
4146509995242,7cyclictest251rcuc/121:27:321
4146509995242,7cyclictest251rcuc/120:48:011
414650499527,16cyclictest1753-21gnome-shell20:10:450
414650499526,16cyclictest1753-21gnome-shell22:38:070
414650499524,18cyclictest1069-21runrttasks20:51:440
414650499522,30cyclictest40280-40lsb_release23:15:220
414650499521,25cyclictest24344-21latency22:30:220
414650499521,25cyclictest24344-21latency22:30:210
162399520,8rtkit-daemon1622-21rtkit-daemon23:58:473
162399520,8rtkit-daemon1622-21rtkit-daemon21:34:360
4146518995110,17cyclictest4182628-21hddtemp_smartct20:50:213
4146515995133,6cyclictest34-21ksoftirqd/221:11:452
4146515995115,9cyclictest34-21ksoftirqd/220:40:172
414651599511,10cyclictest2197014-21PK-Backend23:15:022
4146515995110,21cyclictest34-21ksoftirqd/223:28:562
4146509995142,6cyclictest251rcuc/121:58:011
4146509995119,17cyclictest1753-21gnome-shell22:57:241
414650499518,16cyclictest1753-21gnome-shell22:11:060
414650499517,16cyclictest1753-21gnome-shell23:46:270
414650499517,15cyclictest1753-21gnome-shell21:39:210
414650499516,15cyclictest1753-21gnome-shell19:31:130
414650499515,27cyclictest1069-21runrttasks19:57:580
414650499515,14cyclictest1753-21gnome-shell21:43:360
4146504995110,16cyclictest1753-21gnome-shell21:53:160
4146515995017,11cyclictest34-21ksoftirqd/221:34:592
4146515995016,9cyclictest34-21ksoftirqd/219:54:542
4146515995013,21cyclictest1753-21gnome-shell20:55:532
4146515995012,20cyclictest34-21ksoftirqd/219:26:192
414651599501,10cyclictest2197014-21PK-Backend23:40:012
414650999506,9cyclictest26-21ksoftirqd/123:15:081
4146509995040,7cyclictest251rcuc/123:10:551
4146509995040,7cyclictest251rcuc/122:21:201
4146509995040,7cyclictest251rcuc/121:15:401
4146509995040,7cyclictest251rcuc/119:41:091
4146509995040,7cyclictest251rcuc/119:41:091
4146509995040,7cyclictest251rcuc/100:16:061
414650499508,17cyclictest1753-21gnome-shell21:45:420
414650499507,17cyclictest1753-21gnome-shell22:40:400
414650499507,14cyclictest1753-21gnome-shell21:27:260
414650499506,17cyclictest1753-21gnome-shell23:44:100
414650499506,15cyclictest1753-21gnome-shell23:24:550
414650499506,15cyclictest1753-21gnome-shell23:05:150
414650499506,14cyclictest1753-21gnome-shell22:56:290
414650499506,14cyclictest1753-21gnome-shell22:19:350
414650499502,18cyclictest1753-21gnome-shell20:45:560
414651899497,13cyclictest32131-21taskset22:54:123
414651899497,13cyclictest32131-21taskset22:54:113
4146515994924,5cyclictest141rcu_preempt20:15:282
4146515994913,9cyclictest34-21ksoftirqd/221:51:572
4146515994910,15cyclictest34-21ksoftirqd/222:12:012
4146509994940,7cyclictest251rcuc/120:10:381
4146509994940,7cyclictest251rcuc/119:28:051
4146509994940,6cyclictest251rcuc/122:30:581
4146509994940,6cyclictest251rcuc/122:30:581
4146509994939,7cyclictest251rcuc/121:04:101
4146509994916,16cyclictest1753-21gnome-shell19:59:091
414650499498,16cyclictest1753-21gnome-shell22:50:570
414650499498,16cyclictest1753-21gnome-shell22:50:570
414650499498,16cyclictest1753-21gnome-shell19:12:010
414650499496,18cyclictest1753-21gnome-shell20:06:340
414650499496,16cyclictest1753-21gnome-shell20:22:210
414650499496,13cyclictest61505-21switchtime00:15:310
414650499496,11cyclictest1753-21gnome-shell00:34:110
414650499495,15cyclictest1753-21gnome-shell00:26:310
414650499495,14cyclictest1753-21gnome-shell19:37:520
414650499494,14cyclictest1753-21gnome-shell20:40:340
414651599487,8cyclictest34-21ksoftirqd/200:21:552
414651599483,5cyclictest141rcu_preempt23:40:282
4146515994826,8cyclictest34-21ksoftirqd/220:48:172
414650999489,13cyclictest26-21ksoftirqd/123:20:291
414650999488,17cyclictest26-21ksoftirqd/100:10:151
414650999485,19cyclictest26-21ksoftirqd/122:15:021
4146509994840,6cyclictest251rcuc/100:30:071
4146509994839,7cyclictest251rcuc/119:48:351
4146509994839,6cyclictest251rcuc/123:04:321
4146509994819,10cyclictest26-21ksoftirqd/122:15:151
4146509994816,16cyclictest26-21ksoftirqd/122:41:161
414650499486,15cyclictest1753-21gnome-shell22:28:590
414650499486,14cyclictest1753-21gnome-shell23:58:520
414650499486,14cyclictest1753-21gnome-shell23:25:070
414650499485,15cyclictest1753-21gnome-shell00:08:400
414650499484,14cyclictest1753-21gnome-shell21:20:380
414650499480,30cyclictest12809-21taskset21:56:000
414651899477,8cyclictest4156197-21wc19:35:153
414651599479,18cyclictest34-21ksoftirqd/221:07:132
414651599479,17cyclictest34-21ksoftirqd/220:27:112
4146515994720,5cyclictest141rcu_preempt20:35:462
4146515994710,16cyclictest34-21ksoftirqd/222:25:042
414650999478,11cyclictest26-21ksoftirqd/122:40:011
414650999477,13cyclictest26-21ksoftirqd/121:35:201
414650999476,14cyclictest26-21ksoftirqd/123:52:421
414650999473,8cyclictest26-21ksoftirqd/120:30:011
4146509994737,7cyclictest251rcuc/119:11:321
4146509994726,8cyclictest26-21ksoftirqd/122:25:281
4146509994720,10cyclictest26-21ksoftirqd/100:04:001
4146509994718,10cyclictest26-21ksoftirqd/123:44:251
4146509994716,10cyclictest26-21ksoftirqd/121:51:571
414650499478,17cyclictest4184225-21apt20:55:110
414650499476,19cyclictest4188903-21taskset21:06:270
414650499476,18cyclictest4162861-21taskset19:54:520
414650499476,14cyclictest44945-21taskset23:30:060
414650499476,14cyclictest44945-21taskset23:30:060
414650499475,14cyclictest1753-21gnome-shell22:09:230
414650499475,14cyclictest1753-21gnome-shell22:09:220
414651899467,12cyclictest7413-21sed21:40:323
414651899467,10cyclictest854-21avahi-daemon21:15:243
414651899464,18cyclictest1753-21gnome-shell21:27:363
414651599469,10cyclictest34-21ksoftirqd/221:25:022
414651599467,20cyclictest34-21ksoftirqd/219:36:082
4146515994617,9cyclictest34-21ksoftirqd/220:00:142
4146515994611,18cyclictest34-21ksoftirqd/221:26:442
414650999467,11cyclictest26-21ksoftirqd/120:05:261
414650999466,9cyclictest26-21ksoftirqd/121:30:471
4146509994624,8cyclictest26-21ksoftirqd/120:44:491
4146509994620,9cyclictest26-21ksoftirqd/100:09:531
414650499465,14cyclictest1753-21gnome-shell23:11:070
414651599458,19cyclictest34-21ksoftirqd/222:35:032
414651599458,19cyclictest34-21ksoftirqd/222:35:032
4146515994519,8cyclictest34-21ksoftirqd/200:18:202
4146515994511,14cyclictest141rcu_preempt20:12:242
414651599451,10cyclictest2197014-21PK-Backend20:10:022
414650999459,12cyclictest26-21ksoftirqd/123:10:011
414650999456,11cyclictest26-21ksoftirqd/119:50:201
4146509994536,7cyclictest251rcuc/121:41:231
4146509994535,7cyclictest251rcuc/121:11:031
4146509994525,5cyclictest141rcu_preempt22:01:371
414650999452,39cyclictest1753-21gnome-shell19:30:491
4146509994516,14cyclictest26-21ksoftirqd/120:22:361
4146509994513,9cyclictest26-21ksoftirqd/100:20:241
414650499457,7cyclictest0-21swapper/019:20:240
414650499457,7cyclictest0-21swapper/019:20:230
414650499455,14cyclictest1753-21gnome-shell23:38:530
414650499454,13cyclictest13-21ksoftirqd/019:48:120
141902450,20chrt13-21ksoftirqd/022:00:270
414650999446,9cyclictest26-21ksoftirqd/122:54:441
414650999446,9cyclictest26-21ksoftirqd/122:54:441
414650999446,10cyclictest26-21ksoftirqd/119:39:531
414650999446,10cyclictest26-21ksoftirqd/100:26:321
414650499445,14cyclictest1753-21gnome-shell20:01:110
414650499444,9cyclictest13-21ksoftirqd/022:45:120
4146518994317,11cyclictest42-21ksoftirqd/321:21:043
414651899431,11cyclictest2197014-21PK-Backend19:43:593
414651899431,11cyclictest2197014-21PK-Backend19:43:593
414651599435,12cyclictest141rcu_preempt22:51:252
414651599435,12cyclictest141rcu_preempt22:51:252
4146515994312,16cyclictest4182713-21irqstats20:50:222
414650999437,12cyclictest26-21ksoftirqd/121:05:261
414650999436,10cyclictest26-21ksoftirqd/121:45:441
4146509994316,11cyclictest4177921-21switchtime20:35:321
414650999431,22cyclictest68090-21taskset00:35:051
414650499435,14cyclictest1753-21gnome-shell20:35:180
414651899422,15cyclictest4148032-21taskset19:13:173
4146515994214,16cyclictest34-21ksoftirqd/220:23:392
414650999421,21cyclictest47202-21/usr/sbin/munin23:35:231
4146504994225,12cyclictest438-20systemd-journal20:30:010
414651599415,8cyclictest34-21ksoftirqd/221:41:032
4146515994113,8cyclictest34-21ksoftirqd/219:32:512
414650499412,14cyclictest13-21ksoftirqd/000:22:470
4146515994011,5cyclictest141rcu_preempt21:16:032
414650999404,9cyclictest26-21ksoftirqd/122:50:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional