You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-14 - 03:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Tue Oct 14, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3682910996353,6cyclictest3793461-21sh00:40:013
3682901995525,5cyclictest131rcu_preempt20:15:121
3682905995239,9cyclictest3757641-21idleruntime-cro22:55:002
3682910994938,7cyclictest3734829-21grep21:45:283
3682896994942,4cyclictest3686838-21munin-plugin-st19:20:010
3682905994835,9cyclictest311ktimers/200:30:112
3682896994838,7cyclictest3759331-21cron23:00:010
3682901994737,6cyclictest3700713-21sendmail20:00:001
3682896994737,6cyclictest3731847-21sendmail21:40:010
3682910994639,4cyclictest3769582-21munin-plugin-st23:30:013
3682901994640,4cyclictest3702331-21kworker/u8:2+events_unbound20:42:571
3682905994537,5cyclictest3702331-21kworker/u8:2+events_unbound20:27:562
3682905994537,5cyclictest3702331-21kworker/u8:2+events_unbound20:27:562
3682901994534,7cyclictest3725027-21sendmail21:20:001
3682910994435,5cyclictest3701954-21unixbench_multi20:00:263
3682905994438,4cyclictest3747041-21kworker/u8:3+events_unbound22:33:122
3682901994437,4cyclictest3738645-21cron22:00:001
3682901994432,8cyclictest3713851-21grep20:35:281
3682910994230,9cyclictest3700681-21cron20:00:003
3682905994235,4cyclictest3738645-21cron21:59:592
3682905994235,4cyclictest3707528-21sendmail20:20:002
3682905994133,5cyclictest3769579-21latency_hist23:30:002
3682901994134,5cyclictest3784667-21kworker/u8:2+events_unbound00:13:301
3682896994132,6cyclictest3735647-21fschecks_count21:50:180
3682896994132,6cyclictest3735647-21fschecks_count21:50:180
3682901994031,5cyclictest3688842-21runrttasks19:25:001
3682901994031,5cyclictest3688842-21runrttasks19:25:001
3682901994013,4cyclictest131rcu_preempt20:10:181
3682896994033,4cyclictest3741607-21unixbench_multi22:05:260
3682905993929,6cyclictest3722238-21grep21:05:252
3682905993923,6cyclictest3765873-21kworker/u8:2+events_unbound23:19:392
3682901993927,8cyclictest3765682-21grep23:15:261
3682905993825,5cyclictest131rcu_preempt22:15:022
3682910993727,7cyclictest3747642-21fschecks_time22:25:183
3682905993730,4cyclictest3731830-21sh21:40:002
3682905993725,5cyclictest131rcu_preempt19:57:352
3682905993723,5cyclictest131rcu_preempt20:05:022
3682896993729,5cyclictest3695133-21unixbench_multi19:40:270
3682896993727,6cyclictest0-21swapper/023:25:270
3682905993630,4cyclictest3702331-21kworker/u8:2+events_unbound20:12:532
3682905993618,6cyclictest131rcu_preempt20:30:142
3682905993617,12cyclictest131rcu_preempt19:25:252
3682896993628,4cyclictest0-21swapper/023:05:000
3682910993525,6cyclictest3786199-21grep00:15:273
3682905993527,4cyclictest0-21swapper/223:55:252
3682905993523,5cyclictest131rcu_preempt23:49:042
3682905993517,5cyclictest3735109-21kworker/u8:2-events_freezable_power_21:50:372
3682905993517,5cyclictest3735109-21kworker/u8:2-events_freezable_power_21:50:362
3682905993515,7cyclictest131rcu_preempt20:35:142
3682901993525,5cyclictest0-21swapper/123:40:271
3682901993525,5cyclictest0-21swapper/123:40:271
3682896993525,6cyclictest0-21swapper/020:45:010
3682905993427,4cyclictest0-21swapper/222:40:272
3682905993425,6cyclictest3686839-21grep19:20:002
3682905993425,4cyclictest0-21swapper/221:23:012
3682905993424,7cyclictest3728815-21fschecks_count21:30:142
3682905993424,6cyclictest0-21swapper/221:20:012
3682905993418,5cyclictest131rcu_preempt22:38:502
3682901993424,6cyclictest3764021-21grep23:10:271
3682901993423,8cyclictest440-21cron19:50:001
3682896993428,3cyclictest0-21swapper/020:05:180
3682896993426,4cyclictest0-21swapper/022:35:250
3682896993423,7cyclictest3789582-21grep00:25:260
3682896993423,6cyclictest0-21swapper/023:15:260
3682905993325,4cyclictest0-21swapper/222:00:262
368290599331,18cyclictest0-21swapper/223:40:132
368290599331,18cyclictest0-21swapper/223:40:122
3682905993311,11cyclictest0-21swapper/223:04:402
368290199333,7cyclictest131rcu_preempt22:55:171
3682901993325,5cyclictest0-21swapper/121:30:251
3682896993322,6cyclictest0-21swapper/022:00:280
3682896993316,14cyclictest11-21ksoftirqd/000:35:140
3682910993226,3cyclictest0-21swapper/322:10:253
3682910993224,4cyclictest0-21swapper/320:27:553
3682910993224,4cyclictest0-21swapper/320:27:543
368290599329,7cyclictest32-21ksoftirqd/222:15:282
3682905993221,7cyclictest0-21swapper/200:15:262
3682905993221,5cyclictest131rcu_preempt00:10:002
3682905993215,5cyclictest131rcu_preempt21:00:432
3682905993215,4cyclictest0-21swapper/219:44:082
3682905993211,12cyclictest32-21ksoftirqd/200:25:012
3682905993211,12cyclictest32-21ksoftirqd/200:25:012
3682905993210,8cyclictest32-21ksoftirqd/219:10:182
3682905993210,12cyclictest32-21ksoftirqd/223:35:142
3682901993224,4cyclictest0-21swapper/120:30:271
3682901993224,4cyclictest0-21swapper/120:25:271
3682901993224,4cyclictest0-21swapper/120:25:271
3682901993224,4cyclictest0-21swapper/100:15:161
3682896993226,4cyclictest3784522-21unixbench_singl00:10:260
3682896993223,6cyclictest0-21swapper/021:45:140
3682896993223,5cyclictest0-21swapper/000:20:010
3682896993222,6cyclictest0-21swapper/019:15:010
3682896993222,5cyclictest0-21swapper/020:30:000
3682896993222,5cyclictest0-21swapper/020:30:000
3682896993221,7cyclictest0-21swapper/000:30:260
3682910993122,6cyclictest0-21swapper/323:25:013
3682905993124,5cyclictest3788572-21kworker/u8:2+events_unbound00:28:342
3682905993121,7cyclictest0-21swapper/219:50:302
3682905993117,6cyclictest131rcu_preempt19:35:142
3682905993117,3cyclictest131rcu_preempt20:49:202
3682901993124,3cyclictest0-21swapper/100:30:261
3682901993123,4cyclictest0-21swapper/122:15:191
3682901993120,7cyclictest0-21swapper/119:15:021
3682896993122,6cyclictest3748905-21idleruntime-cro22:30:010
3682896993121,6cyclictest0-21swapper/020:15:270
3682896993121,6cyclictest0-21swapper/019:25:010
3682896993121,6cyclictest0-21swapper/019:25:000
3682910993023,5cyclictest3764148-21kworker/u8:0+events_unbound23:13:203
3682905993023,5cyclictest3752274-21kworker/u8:2+events_unbound22:49:422
3682905993022,6cyclictest3760676-21kworker/u8:0+events_unbound23:05:432
3682905993017,5cyclictest131rcu_preempt23:10:252
3682905993017,3cyclictest131rcu_preempt22:57:592
3682905993015,3cyclictest131rcu_preempt23:52:062
368290599301,16cyclictest32-21ksoftirqd/200:14:592
3682901993022,4cyclictest0-21swapper/121:40:251
3682901993018,8cyclictest3750138-21grep22:30:261
3682896993022,5cyclictest0-21swapper/000:00:000
3682896993019,6cyclictest0-21swapper/021:20:000
3682910992922,4cyclictest0-21swapper/319:15:273
368290599299,12cyclictest32-21ksoftirqd/223:30:272
3682905992921,4cyclictest0-21swapper/220:05:282
3682905992920,6cyclictest301rcuc/219:45:162
3682905992920,4cyclictest0-21swapper/200:35:262
3682905992916,5cyclictest131rcu_preempt19:20:182
3682905992916,5cyclictest131rcu_preempt19:20:172
3682905992915,5cyclictest3702331-21kworker/u8:2+events_unbound20:40:392
3682905992910,11cyclictest32-21ksoftirqd/222:25:262
3682901992923,4cyclictest3691924-21kworker/u8:1+events_unbound19:41:441
3682901992921,5cyclictest0-21swapper/120:00:281
3682901992921,4cyclictest0-21swapper/122:25:271
3682901992919,7cyclictest3742489-21fschecks_time22:10:151
3682901992917,8cyclictest3761047-21apt23:05:001
3682901992912,8cyclictest131rcu_preempt21:20:131
3682896992923,3cyclictest3771274-21debian-sa123:35:000
3682896992922,5cyclictest3690208-21kworker/u8:2+events_unbound19:27:450
3682896992922,4cyclictest3722667-21cut21:10:010
3682896992922,3cyclictest0-21swapper/000:10:000
3682896992921,4cyclictest0-21swapper/023:50:280
3682896992919,6cyclictest0-21swapper/021:00:210
3682896992919,6cyclictest0-21swapper/020:35:190
3682896992918,7cyclictest0-21swapper/020:25:010
3682910992820,4cyclictest0-21swapper/321:15:143
3682910992819,5cyclictest0-21swapper/319:12:423
3682910992817,9cyclictest3790046-21sh00:29:593
368290599289,6cyclictest32-21ksoftirqd/223:20:162
368290599289,13cyclictest131rcu_preempt22:25:012
3682905992813,3cyclictest131rcu_preempt21:10:112
3682905992810,4cyclictest131rcu_preempt22:05:212
3682901992821,4cyclictest0-21swapper/121:35:161
3682901992821,3cyclictest0-21swapper/119:35:011
3682896992822,4cyclictest3756727-21/usr/sbin/munin22:50:240
3682896992821,5cyclictest3691924-21kworker/u8:1+events_unbound19:35:140
3682896992820,4cyclictest0-21swapper/019:45:290
3682896992817,8cyclictest440-21cron23:44:590
3682896992817,8cyclictest440-21cron23:44:590
368289699281,11cyclictest3700286-21grep19:55:270
3682910992720,5cyclictest3688315-21unixbench_multi19:20:273
3682910992720,5cyclictest3688315-21unixbench_multi19:20:273
3682910992720,4cyclictest3774699-21idleruntime-cro23:44:593
3682910992720,4cyclictest3774699-21idleruntime-cro23:44:593
3682910992720,4cyclictest3752305-21sendmail22:40:013
3682910992720,4cyclictest0-21swapper/320:40:003
3682910992719,5cyclictest0-21swapper/323:00:013
3682910992719,4cyclictest0-21swapper/322:45:233
3682910992719,4cyclictest0-21swapper/322:00:273
3682910992716,8cyclictest391ktimers/321:50:153
3682910992716,8cyclictest391ktimers/321:50:153
368290599278,13cyclictest32-21ksoftirqd/220:54:252
368290599277,11cyclictest32-21ksoftirqd/219:30:232
368290599275,7cyclictest32-21ksoftirqd/221:45:232
368290599275,11cyclictest0-21swapper/200:00:202
3682905992713,3cyclictest131rcu_preempt20:23:352
3682901992721,4cyclictest3720742-21kworker/u8:0+events_unbound21:10:461
3682901992720,5cyclictest3702331-21kworker/u8:2+events_unbound20:07:541
3682901992719,5cyclictest3745330-21kworker/u8:0+events_unbound22:21:251
3682896992720,3cyclictest0-21swapper/021:25:250
3682896992719,4cyclictest0-21swapper/020:50:010
3682896992718,5cyclictest0-21swapper/019:55:010
3682910992621,2cyclictest0-21swapper/300:00:013
3682910992619,3cyclictest0-21swapper/323:30:283
3682910992618,5cyclictest0-21swapper/323:50:293
3682910992618,5cyclictest0-21swapper/321:30:243
3682910992618,5cyclictest0-21swapper/320:20:013
3682910992616,7cyclictest3692264-21cat19:35:003
3682905992612,4cyclictest131rcu_preempt21:40:232
3682901992620,4cyclictest3735713-21fschecks_time21:50:191
3682901992620,4cyclictest3735713-21fschecks_time21:50:181
3682901992619,5cyclictest3752274-21kworker/u8:2+events_unbound23:09:281
3682901992618,4cyclictest0-21swapper/122:45:161
3682896992618,5cyclictest0-21swapper/022:35:010
3682896992618,4cyclictest0-21swapper/023:35:250
3682910992519,3cyclictest0-21swapper/321:40:013
3682910992517,4cyclictest0-21swapper/320:35:003
3682910992517,4cyclictest0-21swapper/320:10:013
368290599256,11cyclictest32-21ksoftirqd/221:25:212
3682901992519,3cyclictest0-21swapper/122:05:251
3682901992518,3cyclictest0-21swapper/119:52:481
3682901992517,4cyclictest0-21swapper/123:35:251
3682901992517,4cyclictest0-21swapper/123:30:171
3682901992514,7cyclictest3708776-21grep20:20:291
3682901992510,7cyclictest0-21swapper/121:25:181
3682896992517,5cyclictest0-21swapper/019:30:270
3682896992516,7cyclictest3718402-21kworker/u8:2+events_unbound20:59:230
3682896992516,5cyclictest0-21swapper/022:45:000
3682896992515,6cyclictest0-21swapper/020:10:300
3682910992418,3cyclictest0-21swapper/320:10:213
3682910992416,5cyclictest0-21swapper/322:45:003
3682910992416,5cyclictest0-21swapper/321:05:003
368290599241,15cyclictest0-21swapper/220:56:162
3682901992419,3cyclictest0-21swapper/122:40:201
3682901992418,4cyclictest3755886-21kworker/u8:0+events_unbound22:54:531
3682901992418,3cyclictest0-21swapper/100:40:001
3682901992416,6cyclictest3679074-21kworker/u8:1+events_unbound19:15:171
3682901992416,5cyclictest0-21swapper/120:45:121
3682901992416,4cyclictest0-21swapper/123:25:001
3682896992418,3cyclictest0-21swapper/020:50:250
3682896992418,3cyclictest0-21swapper/000:05:000
3682896992416,5cyclictest0-21swapper/022:45:250
3682896992415,5cyclictest0-21swapper/000:22:070
3682896992415,5cyclictest0-21swapper/000:22:070
368289699241,19cyclictest0-21swapper/023:45:390
368289699241,19cyclictest0-21swapper/023:21:130
368289699241,19cyclictest0-21swapper/022:20:080
368289699241,19cyclictest0-21swapper/022:16:320
368289699241,19cyclictest0-21swapper/020:00:310
368289699240,20cyclictest0-21swapper/021:21:410
3682910992317,3cyclictest0-21swapper/323:05:203
3682910992317,3cyclictest0-21swapper/322:15:213
3682910992316,3cyclictest0-21swapper/323:00:273
3682910992315,5cyclictest0-21swapper/322:55:013
368290199237,8cyclictest131rcu_preempt23:50:161
3682901992316,5cyclictest3782868-21kworker/u8:1+events_unbound00:21:181
3682901992316,5cyclictest3782868-21kworker/u8:1+events_unbound00:21:181
3682901992316,4cyclictest3734825-21head21:45:271
3682901992315,6cyclictest231ktimers/123:25:131
3682896992316,3cyclictest0-21swapper/021:13:570
368289699230,20cyclictest0-21swapper/021:42:300
368289699230,19cyclictest0-21swapper/023:10:410
368289699230,19cyclictest0-21swapper/020:31:050
3682910992216,3cyclictest0-21swapper/323:15:153
3682910992216,3cyclictest0-21swapper/320:55:013
3682910992216,3cyclictest0-21swapper/319:47:483
3682910992216,3cyclictest0-21swapper/300:00:253
3682910992214,5cyclictest391ktimers/320:58:353
3682910992213,6cyclictest3775494-21ntpq23:45:193
368290199225,11cyclictest0-21swapper/119:25:241
3682901992216,4cyclictest3720742-21kworker/u8:0+events_unbound21:04:331
3682901992216,3cyclictest0-21swapper/122:35:151
3682901992212,7cyclictest3693322-21sed19:35:271
3682901992212,5cyclictest3718196-21grep20:50:251
3682901992211,8cyclictest440-21cron00:00:001
368289699220,18cyclictest0-21swapper/021:31:430
368291099219,9cyclictest3693883-21tr19:40:013
3682910992115,4cyclictest3772987-21munin-plugin-st23:39:593
3682910992115,3cyclictest0-21swapper/322:05:213
3682910992115,3cyclictest0-21swapper/321:30:003
3682910992115,3cyclictest0-21swapper/321:05:183
3682910992115,3cyclictest0-21swapper/320:50:013
3682910992115,3cyclictest0-21swapper/319:25:153
3682910992112,6cyclictest0-21swapper/321:55:253
3682910992112,6cyclictest0-21swapper/319:55:023
3682910992111,6cyclictest3726276-21grep21:20:263
3682901992115,3cyclictest0-21swapper/123:45:131
3682901992114,5cyclictest231ktimers/100:00:251
3682901992113,5cyclictest0-21swapper/100:25:271
3682896992110,8cyclictest3742605-21cat22:10:170
368289699210,19cyclictest0-21swapper/021:56:060
3682910992014,4cyclictest391ktimers/321:41:393
3682910992014,3cyclictest0-21swapper/300:14:593
3682910992013,5cyclictest391ktimers/322:30:233
3682910992013,4cyclictest3708071-21if_eth120:20:223
3682910992012,5cyclictest0-21swapper/300:30:163
3682910992010,7cyclictest0-21swapper/322:20:193
368290199205,3cyclictest131rcu_preempt00:05:291
3682901992014,4cyclictest3718402-21kworker/u8:2+events_unbound20:59:361
3682901992013,5cyclictest3736844-21kworker/u8:1+events_unbound22:02:521
368291099199,7cyclictest215-21dbus-daemon21:13:003
3682910991914,3cyclictest3786881-21/usr/sbin/munin00:20:213
3682910991914,3cyclictest3786881-21/usr/sbin/munin00:20:203
3682910991912,4cyclictest0-21swapper/320:45:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional