You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 17:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed Dec 17, 2025 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3478659996452,8cyclictest3522040-21grep09:15:013
3478659995728,25cyclictest0-21swapper/310:50:243
3478644995612,26cyclictest0-21swapper/010:25:200
3478659995427,23cyclictest0-21swapper/308:40:003
3478659995419,7cyclictest0-21swapper/308:10:243
3478659995318,7cyclictest0-21swapper/309:40:253
3478659995225,23cyclictest3568726-21idleruntime-cro11:30:003
3478644995212,27cyclictest3570160-21cron11:31:590
3478644995121,10cyclictest3535132-21grep09:50:220
3478644995121,10cyclictest3535132-21grep09:50:210
3478654995046,2cyclictest3570772-21kworker/u8:0+events_unbound11:38:412
3478654995046,2cyclictest3570772-21kworker/u8:0+events_unbound11:38:412
3478649995042,6cyclictest3487567-21kworker/u8:2+events_unbound08:43:081
3478644995027,15cyclictest0-21swapper/008:50:110
3478649994939,7cyclictest3495984-21grep07:55:261
3478649994939,7cyclictest3495984-21grep07:55:261
3478649994925,6cyclictest131rcu_preempt08:05:111
3478644994938,7cyclictest3558072-21grep10:55:250
3478644994918,9cyclictest3564507-21cut11:15:230
3478644994727,17cyclictest3536117-21/usr/sbin/munin09:55:100
3478644994721,9cyclictest3510856-21ntpq08:40:200
3478659994637,5cyclictest3523949-21audit_send_reply09:20:013
3478659994632,10cyclictest3542704-21sh10:15:013
3478654994622,9cyclictest131rcu_preempt10:50:112
3478649994640,4cyclictest3553001-21kworker/u8:1+events_unbound10:43:321
3478644994627,14cyclictest121ktimers/009:35:110
3478644994616,12cyclictest3482629-21sshd07:18:110
3478659994534,8cyclictest3588976-21grep12:25:253
3478659994534,7cyclictest3584283-21munin-plugin-st12:14:593
3478644994518,9cyclictest3573848-21latency_hist11:45:000
3478644994517,11cyclictest3578383-21ls11:55:240
3478644994515,26cyclictest3513501-21latency_hist08:50:010
3478644994515,26cyclictest3513501-21latency_hist08:50:010
347865999445,4cyclictest34925832tail07:45:253
3478649994437,5cyclictest3563378-21kworker/u8:0+events_unbound11:13:381
3478649994429,7cyclictest3482645-21kworker/u8:1+events_unbound07:42:421
3478644994314,25cyclictest0-21swapper/007:55:260
3478644994314,25cyclictest0-21swapper/007:55:260
3478644994228,11cyclictest3545235-21/usr/sbin/munin10:20:200
3478644994226,11cyclictest3517399-21/usr/sbin/munin09:00:170
3478644994226,11cyclictest3493834-21/usr/sbin/munin07:50:220
3478644994226,11cyclictest3493834-21/usr/sbin/munin07:50:220
3478644994222,16cyclictest3506264-21head08:25:250
3478659994132,6cyclictest3559733-21unixbench_multi11:00:243
3478659994132,5cyclictest3511799-21cron08:45:013
3478654994130,6cyclictest0-21swapper/208:05:242
3478649994134,5cyclictest3487567-21kworker/u8:2+events_unbound07:52:591
3478649994134,5cyclictest3487567-21kworker/u8:2+events_unbound07:52:591
3478644994132,5cyclictest3582371-21munin-plugin-st12:10:000
3478644994128,10cyclictest3504193-21/usr/sbin/munin08:20:210
3478644994122,12cyclictest0-21swapper/009:10:120
3478659994033,4cyclictest3489610-21cron07:40:013
3478659994033,4cyclictest3489610-21cron07:40:003
3478659994032,4cyclictest0-21swapper/309:33:133
3478654994030,6cyclictest0-21swapper/212:00:242
3478654994028,8cyclictest0-21swapper/210:25:232
3478649994028,8cyclictest3510086-21munin-run08:40:011
3478644994031,5cyclictest0-21swapper/008:05:000
3478644994028,9cyclictest3527599-21apt-get09:30:110
3478644994027,8cyclictest0-21swapper/007:10:200
3478644994027,10cyclictest3539539-21/usr/sbin/munin10:05:110
3478644994027,10cyclictest3502227-21/usr/sbin/munin08:15:190
3478644994026,8cyclictest0-21swapper/009:25:110
3478644994026,11cyclictest3570997-21/usr/sbin/munin11:35:180
3478644994026,11cyclictest3570997-21/usr/sbin/munin11:35:170
3478644994026,11cyclictest3558866-21/usr/sbin/munin11:00:150
3478644994026,11cyclictest3488748-21/usr/sbin/munin07:35:230
3478644994026,11cyclictest3488748-21/usr/sbin/munin07:35:230
3478644994026,10cyclictest3483038-21/usr/sbin/munin07:20:120
3478644994021,9cyclictest0-21swapper/009:20:120
3478644994021,14cyclictest3553827-21fschecks_time10:45:150
3478644993932,4cyclictest3551668-21idleruntime-cro10:40:000
3478644993929,7cyclictest3576899-21cron11:51:010
3478644993929,7cyclictest3562128-21/usr/sbin/munin11:10:100
3478644993926,10cyclictest3591782-21missed_timers12:35:180
3478644993926,10cyclictest3588615-21/usr/sbin/munin12:25:230
3478644993923,14cyclictest3541686-21meminfo10:10:190
3478644993916,20cyclictest3500315-21fschecks_time08:10:150
3478654993827,6cyclictest0-21swapper/211:55:252
3478649993830,6cyclictest3529004-21kworker/u8:0+events_unbound09:48:181
3478649993820,5cyclictest131rcu_preempt11:50:201
3478649993811,14cyclictest0-21swapper/108:30:261
3478644993828,7cyclictest3492500-21/usr/sbin/munin07:45:240
3478644993827,7cyclictest3485303-21ntp_states07:25:210
3478644993826,8cyclictest3542941-21/usr/sbin/munin10:15:120
3478644993824,11cyclictest3575469-21sshd11:49:100
3478644993822,13cyclictest3561901-21auditctl11:10:010
3478644993822,12cyclictest3488008-21latency_hist07:35:000
3478644993819,14cyclictest3539487-21cat10:05:010
3478644993819,11cyclictest121ktimers/009:45:110
3478644993818,17cyclictest3590196-21ntpq12:30:200
3478644993816,3cyclictest3586800-21ntp_states12:20:190
3478644993816,3cyclictest3586800-21ntp_states12:20:190
3478644993813,9cyclictest3515421-21chrt08:55:050
3478659993727,6cyclictest3519904-21grep09:05:263
3478654993726,6cyclictest0-21swapper/210:25:012
3478654993726,6cyclictest0-21swapper/209:05:252
3478649993731,4cyclictest3563378-21kworker/u8:0+events_unbound11:23:371
3478649993730,5cyclictest3522465-21kworker/u8:4+events_unbound09:38:151
3478649993714,13cyclictest131rcu_preempt10:05:151
3478644993730,3cyclictest3565677-21diskstats11:20:150
3478644993727,7cyclictest3507218-21/usr/sbin/munin08:30:170
3478644993726,7cyclictest3567375-21/usr/sbin/munin11:25:130
3478644993723,11cyclictest3548925-21/usr/sbin/munin10:30:180
3478644993723,11cyclictest3548925-21/usr/sbin/munin10:30:170
3478644993720,13cyclictest3584961-21cat12:15:180
3478644993720,12cyclictest3532602-21latency_hist09:45:010
3478654993628,6cyclictest3585999-21sendmail12:20:002
3478654993628,4cyclictest0-21swapper/209:25:262
3478654993626,6cyclictest0-21swapper/207:15:232
3478654993625,6cyclictest0-21swapper/212:30:242
3478654993619,5cyclictest3511424-21kworker/u8:1+events_unbound08:56:162
3478649993629,5cyclictest3477175-21kworker/u8:3+events_unbound07:17:541
3478649993627,5cyclictest0-21swapper/107:15:011
3478649993623,3cyclictest131rcu_preempt08:55:261
3478644993630,3cyclictest3582362-21apt-get12:10:050
3478644993629,4cyclictest3518975-21diskstats09:05:160
3478644993626,7cyclictest3579405-21/usr/sbin/munin12:00:140
3478644993626,7cyclictest3556073-21/usr/sbin/munin10:50:230
3478644993626,7cyclictest3522581-21df09:15:130
3478644993626,7cyclictest3508729-21df_inode08:35:150
3478644993625,8cyclictest3490620-21/usr/sbin/munin07:40:240
3478644993621,11cyclictest3551962-21awk10:40:140
3478644993621,11cyclictest3498388-21apt-get08:05:110
3478659993524,6cyclictest3513104-21head08:45:263
3478659993524,6cyclictest3513104-21head08:45:263
3478654993525,7cyclictest0-21swapper/210:20:012
3478654993525,6cyclictest0-21swapper/208:20:012
3478654993524,6cyclictest0-21swapper/209:35:232
3478649993528,4cyclictest3504979-21sh08:25:011
3478649993524,8cyclictest3592316-21grep12:35:231
3478649993517,6cyclictest3532105-21kworker/u8:1+events_unbound10:24:041
3478649993511,5cyclictest131rcu_preempt09:50:111
3478649993511,5cyclictest131rcu_preempt09:50:101
3478659993426,4cyclictest0-21swapper/310:25:013
3478654993427,4cyclictest3554915-21kworker/u8:3+events_unbound11:16:402
3478654993427,3cyclictest0-21swapper/208:35:252
3478654993426,4cyclictest0-21swapper/212:25:252
3478654993424,6cyclictest0-21swapper/211:20:252
3478654993424,6cyclictest0-21swapper/209:20:012
3478649993422,5cyclictest131rcu_preempt12:14:341
3478649993419,3cyclictest131rcu_preempt09:05:171
3478649993418,5cyclictest131rcu_preempt10:00:151
3478649993417,6cyclictest3565234-21kworker/u8:2+events_unbound11:32:051
3478659993325,4cyclictest0-21swapper/309:50:003
3478659993322,7cyclictest0-21swapper/307:50:253
3478659993322,7cyclictest0-21swapper/307:50:253
3478654993326,5cyclictest3549931-21kworker/u8:2+events_unbound10:38:312
3478654993322,5cyclictest3554915-21kworker/u8:3+events_unbound10:59:422
3478654993321,6cyclictest3549931-21kworker/u8:2+events_unbound10:40:242
3478649993327,4cyclictest3521703-21kworker/u8:2+events_unbound09:24:411
3478649993320,5cyclictest3570772-21kworker/u8:0+events_unbound12:08:501
3478649993320,3cyclictest131rcu_preempt11:15:161
3478649993314,7cyclictest3556708-21kworker/u8:1+events_unbound11:29:001
3478649993313,10cyclictest131rcu_preempt08:10:151
347864999331,19cyclictest24-21ksoftirqd/111:05:121
3478659993224,4cyclictest0-21swapper/309:59:593
3478659993223,5cyclictest3549535-21grep10:30:243
3478659993223,5cyclictest3549535-21grep10:30:243
3478659993223,5cyclictest0-21swapper/309:25:163
3478654993226,4cyclictest3532105-21kworker/u8:1+events_unbound10:33:292
3478654993226,4cyclictest3532105-21kworker/u8:1+events_unbound10:33:292
3478654993223,6cyclictest0-21swapper/207:45:242
3478654993223,5cyclictest0-21swapper/208:20:242
347864999328,6cyclictest0-21swapper/107:25:121
3478649993226,4cyclictest3487567-21kworker/u8:2+events_unbound08:53:081
3478659993123,5cyclictest3551667-21sendmail10:40:003
3478659993123,4cyclictest0-21swapper/307:20:253
3478654993125,3cyclictest0-21swapper/209:35:012
3478654993123,4cyclictest0-21swapper/208:33:052
3478654993121,6cyclictest0-21swapper/212:05:252
347865499311,3cyclictest131rcu_preempt11:05:122
347864999319,12cyclictest24-21ksoftirqd/108:15:191
3478649993125,4cyclictest3478634-21cyclictest12:00:001
3478649993123,6cyclictest3580453-21kworker/u8:2+events_unbound12:03:471
3478649993123,6cyclictest3511424-21kworker/u8:1+events_unbound09:33:141
3478649993122,6cyclictest0-21swapper/109:30:011
3478649993122,5cyclictest3558493-21sendmail11:00:011
3478649993115,5cyclictest131rcu_preempt11:40:001
3478649993115,5cyclictest131rcu_preempt11:40:001
3478649993113,6cyclictest24-21ksoftirqd/110:00:001
3478649993110,12cyclictest131rcu_preempt07:40:011
3478649993110,12cyclictest131rcu_preempt07:40:011
3478659993018,9cyclictest3564722-21sed11:15:253
3478659993015,11cyclictest391ktimers/308:00:113
3478654993023,5cyclictest3519967-21kworker/u8:3+events_unbound09:13:122
3478654993013,6cyclictest3532105-21kworker/u8:1+events_unbound10:04:052
3478654993010,6cyclictest3549931-21kworker/u8:2+events_unbound10:48:032
347864999308,12cyclictest24-21ksoftirqd/109:45:011
3478649993021,7cyclictest3482645-21kworker/u8:1+events_unbound07:21:371
3478649993021,5cyclictest0-21swapper/111:45:001
3478649993013,5cyclictest131rcu_preempt11:00:181
3478659992922,4cyclictest0-21swapper/311:50:233
3478659992920,6cyclictest3566564-21unixbench_multi11:20:253
3478659992920,6cyclictest0-21swapper/310:55:263
3478659992919,7cyclictest3578951-21sendmail12:00:013
3478654992922,3cyclictest0-21swapper/207:40:242
3478654992921,6cyclictest3532516-21latency_hist09:45:002
3478654992919,6cyclictest3575134-21grep11:45:232
3478654992917,6cyclictest3570772-21kworker/u8:0+events_unbound12:10:032
3478654992917,6cyclictest3482645-21kworker/u8:1+events_unbound07:39:372
3478654992917,6cyclictest3482645-21kworker/u8:1+events_unbound07:39:372
3478654992916,8cyclictest131rcu_preempt07:55:212
3478654992916,8cyclictest131rcu_preempt07:55:212
3478654992916,6cyclictest3503232-21kworker/u8:3+events_unbound08:28:402
347864999297,4cyclictest131rcu_preempt12:25:171
3478649992923,4cyclictest3568533-21kworker/u8:3+events_unbound12:32:411
3478649992923,4cyclictest3487567-21kworker/u8:2+events_unbound08:28:041
3478649992921,5cyclictest231ktimers/112:15:101
3478649992920,6cyclictest3546786-21cut10:25:141
3478649992917,5cyclictest131rcu_preempt07:30:121
3478649992916,5cyclictest131rcu_preempt10:10:161
347864999291,10cyclictest24-21ksoftirqd/111:45:121
3478649992910,5cyclictest131rcu_preempt09:05:021
3478659992820,4cyclictest0-21swapper/307:40:243
3478659992815,9cyclictest3482818-21basename07:20:013
3478654992820,4cyclictest0-21swapper/208:45:262
3478654992820,4cyclictest0-21swapper/208:45:262
3478654992818,6cyclictest0-21swapper/211:00:252
3478654992815,5cyclictest3568533-21kworker/u8:3+events_unbound11:51:422
347864999289,12cyclictest24-21ksoftirqd/108:50:001
347864999289,12cyclictest24-21ksoftirqd/108:50:001
347864999286,7cyclictest24-21ksoftirqd/110:30:161
347864999286,7cyclictest24-21ksoftirqd/110:30:161
347864999286,11cyclictest24-21ksoftirqd/110:50:001
347864999282,13cyclictest0-21swapper/108:02:121
3478649992820,4cyclictest0-21swapper/110:50:231
3478649992815,5cyclictest3549931-21kworker/u8:2+events_unbound10:37:251
347865999279,3cyclictest3580944-21cpu12:05:133
3478659992721,3cyclictest0-21swapper/310:25:233
3478654992721,4cyclictest3565234-21kworker/u8:2+events_unbound11:33:412
3478654992721,4cyclictest3532105-21kworker/u8:1+events_unbound09:51:422
3478654992721,4cyclictest3532105-21kworker/u8:1+events_unbound09:51:422
3478654992719,4cyclictest0-21swapper/209:45:152
3478654992718,6cyclictest0-21swapper/211:25:152
347865499271,7cyclictest3586439-21grep12:20:132
347865499271,7cyclictest3586439-21grep12:20:132
347865499271,6cyclictest0-21swapper/212:35:112
3478649992718,6cyclictest3491747-21cut07:45:161
3478649992711,5cyclictest131rcu_preempt09:15:221
3478649992711,5cyclictest131rcu_preempt09:10:191
3478659992620,4cyclictest3560205-21latency_hist11:05:013
3478659992619,4cyclictest3544425-21sendmail10:20:013
3478659992617,6cyclictest0-21swapper/308:00:003
3478659992617,6cyclictest0-21swapper/308:00:003
3478659992617,5cyclictest0-21swapper/308:35:003
3478654992620,4cyclictest629-21snmpd09:00:192
3478654992619,5cyclictest3493380-21kworker/u8:0+events_unbound08:00:182
3478654992619,5cyclictest3471764-21kworker/u8:1+events_unbound07:10:042
3478654992616,4cyclictest3493380-21kworker/u8:0+events_unbound07:50:332
3478654992616,4cyclictest3493380-21kworker/u8:0+events_unbound07:50:332
3478654992613,6cyclictest3511424-21kworker/u8:1+events_unbound08:43:342
347864999264,13cyclictest3542940-21/usr/sbin/munin10:15:201
3478649992610,7cyclictest24-21ksoftirqd/112:25:001
3478649992610,7cyclictest24-21ksoftirqd/112:25:001
3478659992518,5cyclictest3568533-21kworker/u8:3+events_unbound12:01:443
3478659992517,4cyclictest0-21swapper/311:30:233
3478659992517,4cyclictest0-21swapper/310:05:223
3478659992516,6cyclictest3516453-21unixbench_multi08:55:253
3478659992516,6cyclictest3503605-21wc08:20:133
347865499255,11cyclictest3486196-21cron07:30:012
347865499253,8cyclictest131rcu_preempt10:10:122
3478654992516,6cyclictest3573419-21grep11:40:232
3478654992515,6cyclictest0-21swapper/207:20:242
3478654992513,4cyclictest131rcu_preempt08:52:492
347865499251,15cyclictest0-21swapper/208:11:232
3478659992418,3cyclictest0-21swapper/312:20:223
3478659992418,3cyclictest0-21swapper/312:20:223
3478659992417,5cyclictest3544342-21kworker/u8:0+events_unbound10:48:323
3478659992416,5cyclictest3571647-21grep11:35:253
3478659992416,5cyclictest3571647-21grep11:35:253
3478654992418,4cyclictest3540544-21unixbench_multi10:05:242
3478654992416,5cyclictest0-21swapper/209:20:262
3478659992316,5cyclictest3484438-21kworker/u8:0+events_unbound07:32:203
3478659992315,5cyclictest0-21swapper/311:40:143
3478659992315,5cyclictest0-21swapper/308:30:003
3478659992315,5cyclictest0-21swapper/308:15:233
3478659992313,7cyclictest3529547-21sort09:35:163
347865499235,7cyclictest131rcu_preempt09:55:192
3478654992315,4cyclictest0-21swapper/211:10:132
347865999222,15cyclictest3591106-21munin-run12:35:003
3478659992216,3cyclictest0-21swapper/312:40:013
3478659992215,5cyclictest164-21systemd-journal09:24:403
3478659992215,4cyclictest0-21swapper/310:40:253
3478659992214,4cyclictest0-21swapper/308:05:253
3478659992213,4cyclictest0-21swapper/309:00:243
3478659992114,5cyclictest391ktimers/309:50:203
3478659992114,5cyclictest391ktimers/309:50:203
3478659992112,6cyclictest0-21swapper/307:25:253
3478659992015,3cyclictest3537999-21fschecks_count10:00:143
3478659992014,3cyclictest0-21swapper/311:45:213
347865999201,14cyclictest3563124-21grep11:10:233
347865999191,13cyclictest3585977-21apt12:19:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional