You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 05:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sun Apr 26, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3949149996556,7cyclictest4095951-21kworker/u8:2+flush-8:021:46:221
3949149996154,5cyclictest20850-21kworker/u8:2+events_unbound23:41:151
3949145996137,21cyclictest4051281-21diskstats20:51:100
3949149995851,5cyclictest3971318-21kworker/u8:0+flush-8:019:56:201
3949149995650,4cyclictest4103405-21kworker/u8:1+flush-8:023:11:201
3949149995650,4cyclictest4103405-21kworker/u8:1+flush-8:023:11:201
3949149995446,6cyclictest45368-21kworker/u8:0+flush-8:000:06:211
3949145995445,6cyclictest70131-21latency_hist00:25:010
3949149995347,4cyclictest4103405-21kworker/u8:1+events_unbound22:51:221
3949149995145,5cyclictest3971318-21kworker/u8:0+flush-8:020:21:201
3949149995143,6cyclictest4103405-21kworker/u8:1+flush-8:023:36:191
3949145995126,22cyclictest4021607-21diskstats20:21:090
3949153995045,4cyclictest4189318-21kworker/u8:3+flush-8:000:01:232
3949149995044,3cyclictest3971318-21kworker/u8:0+events_unbound20:06:081
3949149995041,7cyclictest4154764-21kworker/u8:0+flush-8:022:46:231
41011932490,5chrt0-21swapper/221:41:082
3949149994944,4cyclictest3954095-21kworker/u8:3+flush-8:021:36:211
3949149994943,4cyclictest4124979-21kworker/u8:3+events_unbound22:06:101
3949149994943,4cyclictest3954095-21kworker/u8:3+events_unbound20:26:121
3949149994940,6cyclictest3954095-21kworker/u8:3+flush-8:019:31:211
3949145994939,7cyclictest4125339-21latency_hist22:05:010
3949145994921,17cyclictest4090178-21unixbench_singl21:26:250
3949145994921,17cyclictest4090178-21unixbench_singl21:26:240
403642024841,4sleep30-21swapper/320:36:093
3949153994843,4cyclictest4154764-21kworker/u8:0+flush-8:023:01:232
3949145994831,15cyclictest45358-21basename00:00:000
3949149994742,3cyclictest4189318-21kworker/u8:3+events_unbound23:31:191
3949149994643,2cyclictest3971318-21kworker/u8:0+flush-8:020:01:191
3949149994640,5cyclictest69322-21kworker/u8:2+events_unbound00:26:121
3949149994640,4cyclictest4154764-21kworker/u8:0+flush-8:023:16:091
3949149994640,4cyclictest4154764-21kworker/u8:0+flush-8:023:16:091
3949149994640,4cyclictest3971318-21kworker/u8:0+flush-8:020:41:091
3949149994638,6cyclictest3954095-21kworker/u8:3+events_unbound19:21:201
3949145994637,7cyclictest4085568-21latency_hist21:25:010
3949145994633,10cyclictest3954061-21apt-get19:15:000
3949153994540,4cyclictest3971318-21kworker/u8:0+events_unbound19:46:182
3949149994540,4cyclictest3954095-21kworker/u8:3+flush-8:021:31:131
3949149994539,4cyclictest20850-21kworker/u8:2+events_unbound23:46:171
3949149994536,7cyclictest4154764-21kworker/u8:0+flush-8:023:21:181
3949145994513,22cyclictest0-21swapper/019:21:080
3949153994440,3cyclictest4189318-21kworker/u8:3+events_unbound00:30:012
3949149994438,4cyclictest4124979-21kworker/u8:3+events_unbound22:26:201
3949149994438,4cyclictest4103405-21kworker/u8:1+events_unbound22:01:161
3949149994438,4cyclictest3954095-21kworker/u8:3+flush-8:019:26:191
3949149994437,5cyclictest3971318-21kworker/u8:0+events_unbound19:51:181
3949145994427,15cyclictest3961180-21idleruntime-cro19:20:000
3949145994418,16cyclictest876-21mta-sts-daemon23:13:450
3949145994418,16cyclictest876-21mta-sts-daemon23:13:440
3949149994338,4cyclictest3954095-21kworker/u8:3+events_unbound21:21:191
3949149994336,6cyclictest4095951-21kworker/u8:2+flush-8:022:31:191
3949149994335,6cyclictest4031125-21kworker/u8:1+flush-8:021:16:191
3949145994325,15cyclictest3990483-21ntpq19:46:170
3949149994237,4cyclictest4031125-21kworker/u8:1+events_unbound21:06:181
3949149994236,5cyclictest3971318-21kworker/u8:0+events_unbound20:56:171
3949149994236,4cyclictest4031125-21kworker/u8:1+flush-8:021:01:141
3949149994235,5cyclictest69322-21kworker/u8:2+flush-8:000:36:241
3949149994234,6cyclictest4154764-21kworker/u8:0+flush-8:022:56:201
3949149994233,7cyclictest4144313-21ntpq22:21:181
394914599427,19cyclictest0-21swapper/021:16:080
3949145994213,18cyclictest0-21swapper/023:26:050
394869224228,9sleep10-21swapper/119:06:461
394869224228,9sleep10-21swapper/119:06:461
3949149994137,3cyclictest4103405-21kworker/u8:1+events_unbound22:16:141
3949149994135,4cyclictest4154764-21kworker/u8:0+flush-8:023:01:201
3949149994135,4cyclictest4031125-21kworker/u8:1+events_unbound20:51:221
3949149994134,5cyclictest4189318-21kworker/u8:3+flush-8:023:26:191
3949149994130,8cyclictest280-21systemd-journal20:19:361
3949149994127,10cyclictest45351-40logrotate00:00:001
394914599419,17cyclictest0-21swapper/021:36:090
41905372400,2sleep20-21swapper/223:11:132
41905372400,2sleep20-21swapper/223:11:132
3949149994036,3cyclictest3954095-21kworker/u8:3+flush-8:021:11:231
3949149994034,5cyclictest3971318-21kworker/u8:0+events_unbound20:46:171
394914599403,29cyclictest0-21swapper/000:06:060
3949145994031,7cyclictest4139298-21ntpq22:16:170
394914599399,18cyclictest4036160-21/usr/sbin/munin20:36:240
394914599394,28cyclictest0-21swapper/000:16:070
3949145993924,13cyclictest4021019-21sendmail20:20:000
3949153993834,3cyclictest3971318-21kworker/u8:0+events_unbound20:16:242
3949153993831,5cyclictest280-21systemd-journal19:20:002
3949149993834,2cyclictest4154764-21kworker/u8:0+events_unbound22:36:111
3949149993832,5cyclictest3954095-21kworker/u8:3+flush-8:020:11:211
3949149993832,4cyclictest3986755-21apt-get19:46:051
3949149993832,4cyclictest3954095-21kworker/u8:3+events_unbound20:36:211
3949149993831,5cyclictest4103405-21kworker/u8:1+events_unbound21:55:011
394914599388,18cyclictest589-21runrttasks22:49:390
3949145993822,14cyclictest3986377-21apt-key19:45:000
3949145993822,14cyclictest3986377-21apt-key19:45:000
3949149993733,3cyclictest3971318-21kworker/u8:0+events_unbound20:31:121
3949149993732,3cyclictest3944102-21kworker/u8:2+events_unbound19:11:181
3949149993731,4cyclictest49462-21ntpq00:01:171
394914599374,26cyclictest0-21swapper/023:41:040
3949145993714,21cyclictest4060826-21turbostat.cron21:00:000
3949149993630,5cyclictest4189318-21kworker/u8:3+flush-8:000:21:181
3949149993629,5cyclictest4031125-21kworker/u8:1+flush-8:021:26:101
3949149993629,5cyclictest4031125-21kworker/u8:1+flush-8:021:26:101
3949145993630,4cyclictest4103405-21kworker/u8:1+events_unbound22:21:050
394914599361,11cyclictest4109651-21ntpq21:46:160
3949149993529,5cyclictest4189318-21kworker/u8:3+flush-8:000:16:211
3949149993529,4cyclictest4103405-21kworker/u8:1+events_unbound00:31:111
394914599359,24cyclictest50303-21sshd00:04:360
394914599355,26cyclictest0-21swapper/021:01:060
394914599353,27cyclictest0-21swapper/020:01:050
3949145993529,4cyclictest85019-21gpgv00:40:000
3949145993528,4cyclictest4026349-21grep20:26:070
3949145993517,16cyclictest0-21swapper/023:01:260
394914599344,25cyclictest0-21swapper/023:31:050
394914599344,25cyclictest0-21swapper/019:36:060
394914599344,25cyclictest0-21swapper/019:36:050
3949145993426,6cyclictest35468-21latency_hist23:50:010
394914599341,19cyclictest4134971-21sshd22:11:350
39489352342,3sleep00-21swapper/019:09:540
39489352342,3sleep00-21swapper/019:09:540
3949153993328,4cyclictest4103405-21kworker/u8:1+events_unbound23:57:012
3949149993327,4cyclictest3954095-21kworker/u8:3+flush-8:019:16:211
3949149993326,6cyclictest3971318-21kworker/u8:0+events_unbound19:36:201
3949149993326,6cyclictest3971318-21kworker/u8:0+events_unbound19:36:201
394914599334,2cyclictest0-21swapper/020:46:050
394914599333,3cyclictest0-21swapper/021:51:060
3949145993317,14cyclictest4045117-21mailstats20:41:210
3949145993317,14cyclictest0-21swapper/020:10:010
3949145993316,15cyclictest0-21swapper/023:21:140
3949145993316,14cyclictest0-21swapper/022:06:180
3949153993228,3cyclictest20850-21kworker/u8:2+flush-8:023:51:262
3949153993225,5cyclictest11329-21cut23:26:122
3949149993225,5cyclictest45368-21kworker/u8:0+flush-8:000:11:261
394914599325,5cyclictest4070228-21wc21:06:250
394914599323,3cyclictest0-21swapper/023:16:050
394914599323,3cyclictest0-21swapper/023:16:050
394914599323,2cyclictest0-21swapper/000:11:060
394914599323,26cyclictest0-21swapper/022:51:070
394914599323,21cyclictest0-21swapper/022:31:060
394914599322,3cyclictest0-21swapper/021:41:050
3949145993223,6cyclictest3971091-21latency_hist19:30:010
394914599321,9cyclictest4162483-21/usr/sbin/munin22:41:140
394914599321,8cyclictest4091774-21ps21:31:130
3949145993217,11cyclictest4031125-21kworker/u8:1+flush-8:021:11:070
3949145993215,15cyclictest0-21swapper/000:31:150
3949145993214,16cyclictest0-21swapper/023:51:140
394914599321,10cyclictest4149406-21/usr/sbin/munin22:26:220
3949149993126,3cyclictest4103159-21kthreadcore21:41:141
394914599318,20cyclictest3976331-21date19:35:000
394914599313,4cyclictest0-21swapper/021:56:060
394914599313,3cyclictest0-21swapper/000:26:070
394914599313,24cyclictest0-21swapper/023:06:070
394914599313,24cyclictest0-21swapper/019:51:060
394914599313,23cyclictest0-21swapper/022:56:060
394914599312,6cyclictest3996669-21apt-get19:56:060
394914599311,9cyclictest21479-21/usr/sbin/munin23:36:130
3949145993116,13cyclictest0-21swapper/020:11:090
3949153993026,3cyclictest4124979-21kworker/u8:3+events_unbound22:26:262
3949153993024,4cyclictest4134915-21unixbench_singl22:11:272
3949153993020,8cyclictest4159964-21turbostat.cron22:40:002
3949149993024,4cyclictest4154764-21kworker/u8:0+flush-8:022:41:201
3949149993021,7cyclictest3985369-21ntpq19:41:201
3949149993021,7cyclictest3985369-21ntpq19:41:201
394914599309,18cyclictest4156533-21tr22:36:140
394914599304,2cyclictest0-21swapper/020:31:050
3949153992925,2cyclictest4071106-21aten2_r5power_e21:11:072
3949153992923,4cyclictest3954095-21kworker/u8:3+events_unbound19:25:062
3949149992924,4cyclictest4154764-21kworker/u8:0+flush-8:023:06:211
3949149992923,4cyclictest4115801-21apt-get21:56:051
3949149992923,4cyclictest4103405-21kworker/u8:1+events_unbound23:51:061
3949153992818,8cyclictest4155033-21awk22:35:002
3949153992723,2cyclictest30165-21unixbench_multi23:41:242
3949153992723,2cyclictest21011-21aten2_r5power_e23:36:082
3949153992717,8cyclictest4070724-21awk21:10:002
3949153992623,2cyclictest4006822-21fschecks_count20:06:112
3949153992622,2cyclictest4091203-21cut21:31:102
3949153992621,4cyclictest4103405-21kworker/u8:1+events_unbound00:35:142
3949153992621,3cyclictest4154764-21kworker/u8:0+flush-8:023:15:022
3949153992621,3cyclictest4154764-21kworker/u8:0+flush-8:023:15:012
3949153992620,4cyclictest60237-21cron00:15:002
3949153992522,2cyclictest4110925-21aten2_r5power_e21:51:072
3949153992521,2cyclictest50709-21aten2_r5power_c00:06:072
3949153992521,2cyclictest30971-21aten2_r5power_p23:46:072
3949153992520,3cyclictest3961564-21aten2_r5power_c19:21:082
3949149992420,3cyclictest4095951-21kworker/u8:2+events_unbound22:11:131
40795952230,2sleep3471rcuc/321:16:173
394915399234,7cyclictest0-21swapper/222:56:082
3949153992319,3cyclictest4031125-21kworker/u8:1+events_unbound20:38:212
3949153992319,2cyclictest4096289-21fschecks_count21:36:112
3949153992318,3cyclictest4124954-21grep22:01:232
3949153992313,6cyclictest4061173-21apt-get21:01:072
394915399222,10cyclictest0-21swapper/220:56:062
3949153992219,2cyclictest4026404-21aten2_r5power_v20:26:062
3949153992218,2cyclictest4109607-21needreboot21:46:152
3949153992217,3cyclictest0-21swapper/220:50:002
3949153992216,1cyclictest161rcu_preempt19:11:102
3949153992214,6cyclictest4189588-21turbostat.cron23:09:592
394915399216,3cyclictest41373692cut22:16:132
3949153992116,3cyclictest65747-21cpuspeed_turbos00:21:072
394915399211,2cyclictest161rcu_preempt22:06:262
39444182212,11sleep30-21swapper/319:06:083
39444182212,11sleep30-21swapper/319:06:083
394915399208,2cyclictest4085897-21seq21:26:052
394915399208,2cyclictest4085897-21seq21:26:042
394915399202,5cyclictest768532cat00:31:122
394915399202,4cyclictest0-21swapper/220:51:132
394915399202,2cyclictest161rcu_preempt22:21:242
394915399202,13cyclictest4160318-21apt-get22:41:072
3949153992017,2cyclictest19693-21needreboot23:31:162
3949153992016,2cyclictest3996328-21cron19:55:012
3949153992016,2cyclictest3981203-21cron19:40:002
3949153992016,2cyclictest3981203-21cron19:40:002
3949153992014,4cyclictest0-21swapper/223:21:232
394915399193,7cyclictest0-21swapper/219:41:152
394915399193,7cyclictest0-21swapper/219:41:152
394915399192,16cyclictest0-21swapper/200:16:232
394915399192,12cyclictest4165276-21apt-get22:46:082
3949153991916,2cyclictest4041690-21irqcore20:41:102
394915399190,9cyclictest0-21swapper/222:51:052
39451352194,9sleep20-21swapper/219:06:152
39451352194,9sleep20-21swapper/219:06:152
558722182,6sleep30-21swapper/300:11:103
394915399184,2cyclictest161rcu_preempt20:11:082
394915399183,3cyclictest161rcu_preempt19:56:122
3949153991815,2cyclictest4025618-21unixbench_singl20:21:252
3949153991814,2cyclictest3972141-21cut19:31:122
3949153991814,2cyclictest0-21swapper/220:01:172
39911852172,4sleep30-21swapper/319:47:583
394915399173,2cyclictest161rcu_preempt20:35:002
3949153991713,3cyclictest3954095-21kworker/u8:3+flush-8:021:20:052
3949153991712,3cyclictest4076296-21cut21:16:102
41739472163,10sleep30-21swapper/322:51:193
40858022161,4sleep30-21swapper/321:25:133
40858022161,4sleep30-21swapper/321:25:133
40108792162,5sleep30-21swapper/320:06:523
40010072162,5sleep30-21swapper/319:57:033
3949153991611,3cyclictest0-21swapper/221:56:162
354452162,4sleep30-21swapper/323:49:543
40608002152,5sleep30-21swapper/320:59:533
40258562151,4sleep30-21swapper/320:23:493
41350972141,3sleep30-21swapper/322:13:163
40655602142,4sleep30-21swapper/321:02:403
743122133,6sleep30-21swapper/300:26:233
454982132,6sleep30-21swapper/300:00:023
41936522132,7sleep30-21swapper/323:11:193
41936522132,7sleep30-21swapper/323:11:193
41873412132,7sleep30-21swapper/323:06:173
41639322132,7sleep30-21swapper/322:41:173
41404072132,6sleep30-21swapper/322:20:023
39861782131,7sleep30-21swapper/319:45:013
39861782131,7sleep30-21swapper/319:45:013
206562132,6sleep30-21swapper/323:35:023
810622122,6sleep30-21swapper/300:36:133
756612122,6sleep30-21swapper/300:31:083
545502122,6sleep30-21swapper/300:06:243
52632122,6sleep30-21swapper/323:17:003
52632122,6sleep30-21swapper/323:17:003
41846622122,5sleep30-21swapper/323:05:013
41748132122,5sleep30-21swapper/322:55:013
41655232122,6sleep30-21swapper/322:46:113
41598442122,6sleep30-21swapper/322:38:353
415462122,6sleep30-21swapper/323:56:133
41361032122,6sleep30-21swapper/322:16:133
41253222122,5sleep30-21swapper/322:05:013
41216132122,6sleep30-21swapper/322:01:123
41155512122,6sleep30-21swapper/321:55:023
41051722122,6sleep30-21swapper/321:41:263
40960282122,6sleep30-21swapper/321:36:083
40919632122,6sleep30-21swapper/321:31:153
40312322122,6sleep30-21swapper/320:31:073
40259772122,5sleep30-21swapper/320:25:013
40164272122,6sleep30-21swapper/320:16:073
40119252122,6sleep30-21swapper/320:11:133
40019692122,6sleep30-21swapper/320:01:103
39665112122,6sleep30-21swapper/319:26:073
39537552122,6sleep30-21swapper/319:11:263
158852122,6sleep30-21swapper/323:30:043
662442112,5sleep30-21swapper/300:21:133
609142112,5sleep30-21swapper/300:16:113
56162112,5sleep30-21swapper/323:20:013
41500912111,5sleep30-21swapper/322:30:013
41458912112,5sleep30-21swapper/322:26:123
41148762112,5sleep30-21swapper/321:51:253
41094572112,5sleep30-21swapper/321:46:143
40813092112,5sleep30-21swapper/321:21:113
40715042112,5sleep30-21swapper/321:11:123
40659792112,5sleep30-21swapper/321:05:023
40507232112,4sleep30-21swapper/320:50:013
40502382112,5sleep30-21swapper/320:46:243
40410552112,5sleep30-21swapper/320:40:023
39768532110,7sleep30-21swapper/319:36:103
39768532110,7sleep30-21swapper/319:36:103
39713352112,6sleep30-21swapper/319:30:203
39645392112,5sleep30-21swapper/319:21:173
39568482112,5sleep30-21swapper/319:16:103
361692112,5sleep30-21swapper/323:51:113
259022112,5sleep30-21swapper/323:41:043
108022112,5sleep30-21swapper/323:26:073
39915872102,5sleep30-21swapper/319:50:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional