You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 21:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Mon Nov 17, 2025 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3788222995325,24cyclictest0-21swapper/209:45:282
3788213995041,5cyclictest3891085-21cron12:05:000
3788225994840,4cyclictest3800896-21cron07:45:013
3788222994827,17cyclictest0-21swapper/210:55:182
3788225994736,5cyclictest0-21swapper/309:00:003
3788217994738,5cyclictest3792150-21apt-get07:20:001
3788225994635,8cyclictest391ktimers/310:45:143
3788222994636,7cyclictest3892539-21grep12:05:282
3788222994619,23cyclictest0-21swapper/211:05:192
378822299461,4cyclictest3879376-21apt-get11:30:152
3788213994636,5cyclictest3861867-21cron10:40:010
378822299452,37cyclictest3838012-21apt-get09:30:142
3788222994519,22cyclictest0-21swapper/210:30:002
3788217994538,4cyclictest3879143-21sh11:30:001
3788225994431,9cyclictest440-21cron11:30:013
378822299441,2cyclictest3881077-21apt-get11:35:142
3788217994434,6cyclictest3795306-21grep07:25:281
3788225994332,6cyclictest0-21swapper/312:40:003
3788225994332,6cyclictest0-21swapper/312:39:593
378822299431,8cyclictest3888672-21/usr/sbin/munin11:55:262
378822299431,7cyclictest3805524-21/usr/sbin/munin07:55:282
378822299429,28cyclictest3874728-21sed11:15:222
378822299421,9cyclictest3801769-21/usr/sbin/munin07:45:262
378822299421,8cyclictest3845809-21/usr/sbin/munin09:50:262
378822299421,3cyclictest3802841-21apt-get07:50:162
3788217994229,9cyclictest3901526-21apt12:35:011
3788225994133,5cyclictest3807740-21munin-plugin-st08:05:003
378822299412,35cyclictest3862375-21/usr/sbin/munin10:40:172
378822299412,34cyclictest3903220-21cron12:39:592
378822299412,34cyclictest3903220-21cron12:39:592
378822299411,9cyclictest3876486-21/usr/sbin/munin11:20:202
378822299411,7cyclictest3852634-21swap10:10:292
378822299411,6cyclictest3800009-21ntp_states07:40:252
3788222994116,22cyclictest0-21swapper/212:20:282
378822299411,37cyclictest3893571-21irqstats12:10:192
3788217994110,4cyclictest131rcu_preempt12:20:141
3788213994132,6cyclictest3810013-21fschecks_count08:10:170
3788213994128,10cyclictest121ktimers/010:50:140
3788222994019,18cyclictest0-21swapper/210:00:242
378822299401,3cyclictest3894991-21/usr/sbin/munin12:15:152
378822299401,36cyclictest3860732-21irqstats10:35:192
378822299401,34cyclictest3869777-21/usr/sbin/munin11:00:252
3788217994028,8cyclictest440-21cron07:50:001
3788225993928,8cyclictest391ktimers/312:00:143
378822299393,33cyclictest3833453-21ntp_states09:15:232
378822299392,31cyclictest3882797-21apt-get11:40:142
378822299391,8cyclictest3900463-21/usr/sbin/munin12:30:212
378822299391,7cyclictest3885372-21/usr/sbin/munin11:45:262
378822299391,7cyclictest3885372-21/usr/sbin/munin11:45:262
378822299391,7cyclictest3818042-21sshd08:33:362
378822299391,5cyclictest3831323-21/usr/sbin/munin09:10:182
378822299391,4cyclictest0-21swapper/211:25:142
378822299391,3cyclictest3850288-21diskstats10:05:182
378822299391,3cyclictest0-21swapper/208:05:152
378822299391,35cyclictest3793990-21cron07:24:012
378822299391,33cyclictest3830872-21latency_hist09:10:012
378822299391,33cyclictest3820223-21df08:40:152
378822299391,32cyclictest3889834-21cut12:00:172
378822299390,3cyclictest0-21swapper/208:20:142
3788217993929,6cyclictest3812571-21grep08:15:301
3788217993928,7cyclictest3863592-21cut10:45:011
3788217993928,7cyclictest3793684-21grep07:20:271
3788213993931,5cyclictest3814313-21unixbench_multi08:20:280
3788213993930,6cyclictest3877009-21unixbench_singl11:20:260
378822299382,30cyclictest3788376-21apt-get07:10:122
378822299381,9cyclictest3827109-21apt-get09:00:062
378822299381,5cyclictest3846985-21meminfo09:55:232
378822299381,5cyclictest3836424-21df_inode09:25:152
378822299381,4cyclictest3812097-21/usr/sbin/munin08:15:262
378822299381,4cyclictest3797899-21/usr/sbin/munin07:35:172
378822299381,4cyclictest3790795-21diskstats07:15:152
378822299381,3cyclictest3840789-21/usr/sbin/munin09:35:272
378822299381,3cyclictest3818402-21apt-get08:35:132
378822299381,3cyclictest0-21swapper/208:10:152
378822299381,3cyclictest0-21swapper/207:30:132
378822299381,34cyclictest3815115-21df_inode08:25:162
378822299381,32cyclictest3898808-21meminfo12:25:202
378822299381,32cyclictest3835186-21ntp_states09:20:252
378822299381,10cyclictest3865383-21grep10:50:012
378822299381,10cyclictest3826086-21/usr/sbin/munin08:55:192
378822299380,8cyclictest3823850-21entropy08:50:162
378822299380,3cyclictest0-21swapper/208:45:142
3788217993827,7cyclictest3852883-21grep10:10:291
3788213993831,4cyclictest3799192-21idleruntime-cro07:40:010
3788213993818,3cyclictest0-21swapper/010:10:010
3788225993731,4cyclictest3821437-21kworker/u8:1+events_unbound08:46:013
3788225993726,7cyclictest0-21swapper/312:10:003
378822299373,31cyclictest3873116-21ntp_states11:10:222
378822299372,30cyclictest3860150-21cron10:35:002
378822299371,9cyclictest3856413-21cron10:21:002
378822299371,9cyclictest3856413-21cron10:21:002
378822299371,4cyclictest3841719-21/usr/sbin/munin09:40:192
378822299371,3cyclictest3806781-21/usr/sbin/munin08:00:212
378822299371,32cyclictest3887245-21grep11:50:272
378822299371,31cyclictest3865850-21irqstats10:50:192
3788213993730,4cyclictest3827119-21idleruntime-cro09:00:010
3788213993726,6cyclictest0-21swapper/011:40:250
3788225993626,6cyclictest0-21swapper/309:45:173
3788225993626,6cyclictest0-21swapper/309:45:013
3788225993625,7cyclictest0-21swapper/310:05:303
3788225993625,6cyclictest0-21swapper/307:45:283
378822299361,5cyclictest3794314-21/usr/sbin/munin07:25:132
378822299361,3cyclictest3854998-21sshd10:19:362
3788217993628,4cyclictest0-21swapper/107:40:001
3788213993630,4cyclictest3841189-21munin-plugin-st09:40:020
3788213993628,4cyclictest0-21swapper/010:10:300
3788225993525,6cyclictest0-21swapper/312:35:003
3788225993525,6cyclictest0-21swapper/312:15:003
3788225993524,7cyclictest0-21swapper/311:30:183
3788217993524,6cyclictest0-21swapper/109:45:271
378821399359,6cyclictest11-21ksoftirqd/009:20:000
3788213993527,5cyclictest3845828-21unixbench_multi09:50:260
3788213993527,3cyclictest0-21swapper/008:20:010
3788225993427,4cyclictest3861894-21idleruntime-cro10:40:013
3788225993426,4cyclictest0-21swapper/310:55:293
3788225993425,6cyclictest0-21swapper/307:55:003
3788225993425,5cyclictest0-21swapper/308:20:013
3788217993425,4cyclictest0-21swapper/110:20:301
3788217993425,4cyclictest0-21swapper/110:20:291
3788217993424,7cyclictest3868916-21sendmail11:00:001
3788213993424,6cyclictest3868422-21unixbench_multi10:55:290
3788213993423,6cyclictest0-21swapper/011:50:170
3788225993326,5cyclictest3897986-21kworker/u8:1+events_unbound12:26:343
3788225993323,6cyclictest0-21swapper/311:45:183
3788225993323,6cyclictest0-21swapper/311:45:173
3788217993325,4cyclictest0-21swapper/108:55:271
3788213993326,3cyclictest0-21swapper/007:20:000
3788213993325,6cyclictest3870961-21kworker/u8:3+events_unbound11:16:250
3788213993323,6cyclictest0-21swapper/012:05:280
3788213993323,6cyclictest0-21swapper/011:35:160
3788225993225,3cyclictest0-21swapper/311:01:233
3788225993224,5cyclictest3786043-21kworker/u8:1+events_unbound07:15:263
3788217993225,5cyclictest3884044-21kworker/u8:0+events_unbound11:56:321
3788217993224,5cyclictest3830901-21latency_hist09:10:011
3788217993224,4cyclictest0-21swapper/110:05:001
3788217993222,6cyclictest0-21swapper/112:25:261
3788217993211,4cyclictest131rcu_preempt10:25:131
3788213993225,4cyclictest3856751-21cron10:25:000
3788213993225,4cyclictest3856751-21cron10:25:000
3788213993225,3cyclictest0-21swapper/010:20:000
3788213993225,3cyclictest0-21swapper/010:05:010
3788213993224,5cyclictest3872308-21sh11:10:000
3788213993222,6cyclictest0-21swapper/011:55:260
3788225993123,4cyclictest3788534-21fschecks_count07:10:183
378821799315,4cyclictest131rcu_preempt08:35:131
3788217993124,4cyclictest0-21swapper/110:05:191
3788217993123,4cyclictest0-21swapper/108:10:261
3788217993121,6cyclictest3887211-21grep11:50:271
3788217993121,6cyclictest0-21swapper/111:30:271
3788217993120,6cyclictest0-21swapper/110:40:001
3788213993124,3cyclictest0-21swapper/009:20:200
3788213993121,7cyclictest121ktimers/012:20:150
3788213993121,6cyclictest0-21swapper/009:30:000
3788213993120,7cyclictest0-21swapper/008:05:270
3788213993110,5cyclictest11-21ksoftirqd/009:45:190
3788225993024,3cyclictest3819386-21tail08:35:273
3788225993023,4cyclictest3863593-21idleruntime-cro10:45:003
3788225993022,4cyclictest0-21swapper/312:20:013
3788225993021,4cyclictest0-21swapper/307:20:253
3788225993020,5cyclictest3859679-21tail10:30:273
3788217993024,3cyclictest0-21swapper/109:15:011
3788217993023,5cyclictest3901746-21kworker/u8:2+events_unbound12:36:361
3788217993023,5cyclictest3901746-21kworker/u8:2+events_unbound12:36:361
3788217993022,5cyclictest3861684-21kworker/u8:2+events_unbound11:05:161
3788217993022,4cyclictest0-21swapper/111:20:251
3788217993020,6cyclictest0-21swapper/110:30:261
3788217993019,7cyclictest3894697-21sh12:15:001
3788217993019,7cyclictest3892997-21sh12:10:011
3788217993017,8cyclictest215-21dbus-daemon08:46:001
3788213993025,3cyclictest11-21ksoftirqd/008:30:260
3788213993020,6cyclictest0-21swapper/010:30:260
3788225992922,3cyclictest0-21swapper/310:20:293
3788225992922,3cyclictest0-21swapper/310:20:293
3788225992921,4cyclictest0-21swapper/311:10:253
3788225992921,4cyclictest0-21swapper/308:45:013
3788225992921,3cyclictest0-21swapper/307:40:003
3788225992919,6cyclictest3866979-21auditctl10:55:003
3788225992919,6cyclictest0-21swapper/308:10:023
3788225992917,8cyclictest1-21systemd12:21:333
3788217992920,6cyclictest3845036-21tune2fs09:50:181
3788217992919,6cyclictest0-21swapper/107:40:291
3788213992921,4cyclictest0-21swapper/008:46:000
3788213992920,6cyclictest3816001-21grep08:25:280
3788213992920,5cyclictest0-21swapper/008:05:010
3788213992919,6cyclictest0-21swapper/010:25:230
3788225992820,4cyclictest0-21swapper/309:35:283
3788225992819,5cyclictest0-21swapper/308:30:003
3788225992814,9cyclictest3834366-21munin-plugin-st09:20:003
3788217992819,5cyclictest0-21swapper/111:15:291
3788217992819,4cyclictest0-21swapper/111:40:001
378821399285,16cyclictest0-21swapper/009:05:190
3788213992820,4cyclictest0-21swapper/012:15:270
3788213992820,4cyclictest0-21swapper/009:40:170
3788213992818,6cyclictest0-21swapper/011:00:280
3788213992816,8cyclictest215-21dbus-daemon12:36:350
3788213992816,8cyclictest215-21dbus-daemon12:36:340
3788225992721,4cyclictest3870961-21kworker/u8:3+events_unbound11:36:303
3788225992720,4cyclictest3845741-21timerandwakeup09:50:263
3788225992720,3cyclictest0-21swapper/311:50:173
3788225992720,3cyclictest0-21swapper/311:15:263
3788225992720,3cyclictest0-21swapper/309:20:293
3788225992718,5cyclictest0-21swapper/307:35:003
3788225992717,6cyclictest0-21swapper/311:55:213
3788225992716,7cyclictest0-21swapper/309:55:293
378821399277,6cyclictest0-21swapper/007:50:150
3788213992717,6cyclictest0-21swapper/008:35:270
3788213992716,6cyclictest0-21swapper/007:10:280
3788213992710,5cyclictest11-21ksoftirqd/011:25:140
3788225992619,4cyclictest3852924-21unixbench_singl10:10:303
3788225992618,5cyclictest3855034-21cron10:20:003
3788225992617,5cyclictest0-21swapper/308:10:263
3788217992620,3cyclictest3788206-21cyclictest08:40:121
3788217992618,5cyclictest0-21swapper/110:45:281
3788217992618,5cyclictest0-21swapper/109:30:261
3788217992618,4cyclictest0-21swapper/108:30:171
3788217992616,7cyclictest0-21swapper/111:10:191
378821399269,12cyclictest3831711-21ntpq09:10:220
3788213992619,4cyclictest0-21swapper/012:25:260
3788213992619,3cyclictest0-21swapper/010:41:560
378821399261,6cyclictest3828024-21memory09:00:240
3788213992615,6cyclictest131rcu_preempt07:43:560
3788213992615,5cyclictest131rcu_preempt10:47:500
3788217992518,4cyclictest24-21ksoftirqd/108:00:121
3788217992518,3cyclictest3790061-21latency_hist07:15:011
3788217992517,5cyclictest0-21swapper/107:35:011
3788217992516,6cyclictest3839918-21grep09:35:181
3788217992516,5cyclictest0-21swapper/111:45:261
3788217992516,5cyclictest0-21swapper/111:45:261
378821399259,7cyclictest0-21swapper/007:31:560
378821399254,6cyclictest0-21swapper/007:25:270
3788213992517,5cyclictest0-21swapper/011:10:160
3788213992513,6cyclictest131rcu_preempt07:58:140
3788213992510,8cyclictest131rcu_preempt09:55:030
3788225992418,3cyclictest0-21swapper/309:25:163
3788225992418,3cyclictest0-21swapper/308:20:193
3788225992416,5cyclictest0-21swapper/310:05:013
3788217992419,3cyclictest3814538-21kworker/u8:1-writeback08:25:171
3788217992417,3cyclictest0-21swapper/112:15:171
3788217992414,6cyclictest3804297-21cron07:55:001
3788217992413,7cyclictest0-21swapper/108:55:021
378821799241,11cyclictest0-21swapper/109:15:181
378821399248,3cyclictest131rcu_preempt07:47:170
378821399245,16cyclictest3792870-21basename07:20:180
3788213992411,2cyclictest0-21swapper/008:51:030
3788225992316,5cyclictest391ktimers/307:25:243
3788225992315,5cyclictest0-21swapper/310:30:013
3788225992315,5cyclictest0-21swapper/309:35:003
3788225992315,5cyclictest0-21swapper/309:05:263
378822599231,12cyclictest0-21swapper/311:20:203
3788217992317,4cyclictest3882318-21kworker/u8:2+events_unbound11:43:441
3788217992317,4cyclictest3807385-21kworker/u8:1+events_unbound08:05:461
3788217992316,5cyclictest3835937-21kworker/u8:2+events_unbound09:26:121
3788217992316,3cyclictest0-21swapper/107:55:271
3788217992315,5cyclictest3851424-21kworker/u8:2+events_unbound10:16:431
3788213992316,3cyclictest0-21swapper/012:10:060
378821399231,15cyclictest0-21swapper/011:48:150
378821399231,15cyclictest0-21swapper/011:48:140
378821399230,19cyclictest0-21swapper/012:30:570
3788225992216,4cyclictest146-21jbd2/mmcblk1p2-808:30:243
3788225992215,4cyclictest3872304-21cron11:10:003
3788225992215,4cyclictest3824278-21ntpq08:50:213
3788225992211,8cyclictest3806016-21cat08:00:003
378821799222,5cyclictest131rcu_preempt10:50:131
3788217992216,3cyclictest0-21swapper/109:00:191
3788217992215,4cyclictest0-21swapper/109:40:271
3788217992215,3cyclictest0-21swapper/108:20:271
3788217992214,5cyclictest3846092-21kworker/u8:2+events_unbound09:56:531
3788217992213,5cyclictest0-21swapper/112:05:011
378821399227,4cyclictest131rcu_preempt11:30:240
3788225992113,5cyclictest0-21swapper/309:00:303
3788225992111,6cyclictest0-21swapper/311:40:243
3788217992113,4cyclictest0-21swapper/109:25:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional